ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Floop: An efficient video coding flow for unmanned aerial vehicles

  • Yu Su (China Mobile Chengdu Institute of Research and Development) ;
  • Qianqian Cheng (China Mobile Chengdu Institute of Research and Development) ;
  • Shuijie Wang (China Mobile Chengdu Institute of Research and Development) ;
  • Jian Zhou (China Mobile Chengdu Institute of Research and Development) ;
  • Yuhe Qiu (China Mobile Chengdu Institute of Research and Development)
  • Received : 2022.03.18
  • Accepted : 2022.08.15
  • Published : 2023.08.10
Download PDF
⟨ Previous Next ⟩

Abstract

Under limited transmission conditions, many factors affect the efficiency of video transmission. During the flight of an unmanned aerial vehicle (UAV), frequent network switching often occurs, and the channel transmission condition changes rapidly, resulting in low-video transmission efficiency. This paper presents an efficient video coding flow for UAVs working in the 5G nonstandalone network and proposes two bit controllers, including time and spatial bit controllers, in the flow. When the environment fluctuates significantly, the time bit controller adjusts the depth of the recursive codec to reduce the error propagation caused by excessive network inference. The spatial bit controller combines the spatial bit mask with the channel quality multiplier to adjust the bit allocation in space to allocate resources better and improve the efficiency of information carrying. In the spatial bit controller, a flexible mini graph is proposed to compute the channel quality multiplier. In this study, two bit controllers with end-to-end codec were combined, thereby constructing an efficient video coding flow. Many experiments have been performed in various environments. Concerning the multi-scale structural similarity index and peak signal-to-noise ratio, the performance of the coding flow is close to that of H.265 in the low bits per pixel area. With an increase in bits per pixel, the saturation bottleneck of the coding flow is at the same level as that of H.264.

Keywords

Acknowledgement

We would like to thank Yuanlin Xu, Ziyang Liu, Yike Ren, and Hangyu Li and othe for UAV data collection. This work is supported by the CMLC team of China Mobile Chengdu Institute of Research and Development, Chengdu, China.

References

  1. A. Ogawa, S. Kuroda, K. Ushida, R. Kudo, K. Tateishi, H. Yamashita, and T. Kantou, Field experiments on sensor data transmission for 5G-based vehicle-infrastructure cooperation, (IEEE 88th Vehicular Technology Conference, Chicago, IL, USA), 2018, pp. 1-5.
  2. T. Higuchi, N. Shimizu, H. Shingu, Y. Morihiro, Y. Okumura, T. Miyagoshi, and H. Asano, Video sending rate prediction based on communication logging database for 5G HetNet, (IEEE 87th Vehicular Technology Conference, Chicago, IL, USA), 2018, pp. 1-6.
  3. B. Li, Z. Fei, and Y. Zhang, UAV communications for 5G and beyond: Recent advances and future trends, IEEE Int. Things J. 6 (2018), no. 2, 2241-2263.
  4. X. Hu, B. Pang, F. Dai, and K. H. Low, Risk assessment model for UAV cost-effective path planning in urban environments, IEEE Access 8 (2020), 150162-150173. https://doi.org/10.1109/ACCESS.2020.3016118
  5. P. Tokekar, J. Vander Hook, D. Mulla, and V. Isler, Sensor planning for a symbiotic UAV and UGV system for precision agriculture, IEEE Trans. Robot. 32 (2016), no. 6, 1498-1511. https://doi.org/10.1109/TRO.2016.2603528
  6. N. Chen, N. Ni, P. Kapp, J. Chen, A. Xiao, and H. Li, Structural analysis of the Hero Range in the Qaidam Basin, northwestern China, using integrated UAV, terrestrial LiDAR, Landsat 8, and 3-D seismic data, IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 8 (2015), no. 9, 4581-4591. https://doi.org/10.1109/JSTARS.2015.2440171
  7. L. Wirth, P. Oettershagen, J. Ambuhl, and R. Siegwart, Meteorological path planning using dynamic programming for a solar-powered UAV, (IEEE Aerospace Conference, Big SKY, MT, USA), 2015, pp. 1-11.
  8. Q. Yuan, Q. Qian, Y. Mo, and H. Chen, Research on Mixed Planning Method of 5G and LTE, (3rd International Conference on Information and Computer Technologies, San Jose, CA, USA), 2020, pp. 489-493.
  9. Verizon, What is millimeter wave technology? June 2018. https://www.verizon.com/about/our-company/5G/whatmillimeter-wave-technology
  10. T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Overview of the H. 264/AVC video coding standard, IEEE Trans. Circ. Syst. Video Technol. 13 (2003), no. 7, 560-576. https://doi.org/10.1109/TCSVT.2003.815165
  11. H. Schwarz, D. Marpe, and T. Wiegand, Overview of the Scalable Video Coding Extension of the H.264/AVC Standard, IEEE Trans. Circ. Syst. Video Technol. 17 (2007), no. 9, 1103-1120. https://doi.org/10.1109/TCSVT.2007.905532
  12. G. J. Sullivan, J. R. Ohm, W. J. Han, and T. Wiegand, Overview of the high efficiency video coding (HEVC) standard, IEEE Trans. Circ. Syst. Video Technol. 22 (2012), no. 12, 1649-1668. https://doi.org/10.1109/TCSVT.2012.2221191
  13. J. R. Ohm, G. J. Sullivan, H. Schwarz, T. K. Tan, and T. Wiegand, Comparison of the coding efficiency of video coding standards-including high efficiency video coding (HEVC), IEEE Trans. Circ. Syst. Video Technol. 22 (2012), no. 12, 1669-1684. https://doi.org/10.1109/TCSVT.2012.2221192
  14. G. Lu, W. Ouyang, D. Xu, X. Zhang, C. Cai, and Z. Gao, DVC: An end-to-end deep video compression framework, (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA), 2019, pp. 11006-11015.
  15. S. Huo, D. Liu, F. Wu, and H. Li, Convolutional neural network-based motion compensation refinement for video coding, (IEEE International Symposium on Circuits and Systems, Florence, Italy), 2018, pp. 1-4.
  16. C. Y. Wu, N. Singhal, and P. Krahenbuhl, Video compression through image interpolation, (Proceedings of the European Conference on Computer Vision, Munich, German), 2018, pp. 416-431.
  17. G. Toderici, D. Vincent, N. Johnston, S. Jin Hwang, D. Minnen, J. Shor, and M. Covell, Full resolution image compression with recurrent neural networks, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA), 2017, pp. 5306-5314.
  18. Z. Chen, T. He, X. Jin, and F. Wu, Learning for video compression, IEEE Trans. Circ. Syst. Video Technol. 30 (2019), no. 2, 566-576.
  19. J. H. Hu, W. H. Peng, and C. H. Chung, Reinforcement learning for HEVC/H. 265 intra-frame rate control, (IEEE International Symposium on Circuits and Systems, Florence, Italy), 2018, pp. 1-5.
  20. O. Rippel, S. Nair, C. Lew, S. Branson, A. G. Anderson, and L. Bourdev, Learned video compression, (Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Rep. of Korea), 2019, pp. 3454-3463.
  21. A. Tsakmalis, S. Chatzinotas, and B. Ottersten, Modulation and coding classification for adaptive power control in 5G cognitive communications, (IEEE 15th International Workshop on Signal Processing Advances in Wireless Communications, Toronto, Canada), 2014, pp. 234-238.
  22. I. O. Sanusi, K. M. Nasr, and K. Moessner, Channel assignment and power control for D2D-enabled cellular networks, (International Conference on Computing, Electronics & Communications Engineering, London, UK), 2019, pp. 225-228.
  23. J. Yeo, S. Park, J. Oh, Y. Kim, and J. Lee, Partial retransmission scheme for HARQ enhancement in 5G wireless communications, (IEEE GLOBECOM Workshops, Singapore), 2017, pp. 1-5.
  24. B. Wang, Y. Zhu, and J. Kang, Two effective scheduling schemes for layered belief propagation of 5G LDPC codes, IEEE Commun. Lett. 24 (2020), no. 8, 1683-1686. https://doi.org/10.1109/LCOMM.2020.2991473
  25. F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. Van Gool, Conditional probability models for deep image compression, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA), 2018, pp. 4394-4402.
  26. D. Minnen, J. Balle, and G. D. Toderici, Joint autoregressive and hierarchical priors for learned image compression, Adv. Neural Inform. Process. Syst. (2018), 31.
  27. Q. Gao, M. Ji, L. Pang, W. T. Jiang, P. Fan, and X. Zhang, Design of UAV high resolution image transmission system, (International Conference on Optical and Photonics Engineering, Chengdu, China), 2017, pp. 591-601.
  28. M. Bhaskaranand and J. D. Gibson, Low-complexity video encoding for UAV reconnaissance and surveillance, (MILCOM 2011 Military Communications Conference, Baltimore, USA), 2011, pp. 1633-1638.
  29. M. Bhaskaranand and J. D. Gibson, Global motion assisted low complexity video encoding for UAV applications, IEEE J. Sel. Top. Signal Process. 9 (2014), no. 1, 139-150. https://doi.org/10.1109/JSTSP.2014.2345563
  30. X. Chen, S. Zhang, and J. Liu, Design of UAV video compression system based on H. 264 encoding algorithm, (Proceedings of 2011 International Conference on Electronic & Mechanical Engineering and Information Technology, Harbin, China), 2011, pp. 2619-2622.
  31. H. Meuel, M. Munderloh, and J. Ostermann, Low bit rate ROI based video coding for HDTV aerial surveillance video sequences, (CVPR 2011 WORKSHOPS, Colorado Springs, CO. USA), 2011, pp. 13-20.
  32. M. Naveed, S. Qazi, B. A. Khawaja, and M. Mustaqim, Evaluation of video streaming capacity of UAVs with respect to channel variation in 4G-LTE Surveillance Architecture, (8th International Conference on Information and Communication Technologies, Karachi, Pakistan), 2019, pp. 149-154.
  33. D. L. Hench, P. N. Topiwala, and Z. Xiong, Channel adaptive video compression for unmanned aerial vehicles (UAVs), (the SPIE Annual Meeting Optical Science and Technology, Denver. CO. USA), 2004, pp. 475-484.