DOI QR코드

DOI QR Code

Improvement of Frame Rate of Electro-Optical Sensor using Temporal Super Resolution based on Color Channel Extrapolation

채널별 색상정보 외삽법 기반 시간적 초해상도 기법을 활용한 전자광학 센서의 프레임률 향상 연구

  • Received : 2017.01.20
  • Accepted : 2017.05.12
  • Published : 2017.05.31

Abstract

The temporal super resolution is a method for increasing the frame rate. Electro-optical sensors are used in various surveillance and reconnaissance weapons systems, and the spatial resolution and temporal resolution of the required electro-optical sensors vary according to the performance requirement of each weapon system. Because most image sensors capture images at 30~60 frames/second, it is necessary to increase the frame rate when the target moves and changes rapidly. This paper proposes a method to increase the frame rate using color channel extrapolation. Using a DMD, one frame of a general camera was adjusted to have different consecutive exposure times for each channel, and the captured image was converted to a single channel image with an increased frame rate. Using the optical flow method, a virtual channel image was generated for each channel, and a single channel image with an increased frame rate was converted to a color channel image. The performance of the proposed temporal super resolution method was confirmed by the simulation.

Keywords

Color channel extrapolation;Dynamic micromirror device;Frame rate;Oprical flow;Temporal super resolution

References

  1. D. Mahajan, F. C. Huang, W. Matusik, R. Ramamoorthi, and P. Belhumeur, "Moving gradients: a path-based method for plausible image interpolation." Proceedings of SIGGRAPH, 2009. DOI: https://doi.org/10.1145/1531326.1531348 https://doi.org/10.1145/1531326.1531348
  2. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High accuracy optical flow estimation based on a theory for warping." European Conference on Computer Vision (ECCV), pp. 25-36, 2004. DOI: https://doi.org/10.1007/978-3-540-24673-2_3 https://doi.org/10.1007/978-3-540-24673-2_3
  3. C. Wang, L. Zhang, Y. He, and Y. P. Tan, "Frame rate up-conversion using trilateral filtering." IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Vol. 20, Issue. 6, pp. 886-893, 2010. DOI: https://doi.org/10.1109/TCSVT.2010.2046057 https://doi.org/10.1109/TCSVT.2010.2046057
  4. G. Bub, M. Tecza, M. Helmes, P. Lee, and P. Kohl, "Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging." Nature methods, Vol. 7, Issue. 3, pp. 209-211, 2010. DOI: https://doi.org/10.1038/nmeth.1429 https://doi.org/10.1038/nmeth.1429
  5. J. Gu, Y. Hitomi, T. Mitsunaga, and S. Nayar, "Coded rolling shutter photography: Flexible space-time sampling." IEEE International Conference on Computational Photography (ICCP), 2010. DOI: https://doi.org/10.1109/ICCPHOT.2010.5585094 https://doi.org/10.1109/ICCPHOT.2010.5585094
  6. D. Reddy, A. Veeraraghavan, and R. Chellappa, "P2C2: Programmable pixel compressive camera for high speed imaging." Computer Vision and Pattern Recognition (CVPR), 2011. DOI: https://doi.org/10.1109/CVPR.2011.5995542 https://doi.org/10.1109/CVPR.2011.5995542
  7. D. Liu, J. Gu, Y. Hitomi, M. Gupta, T. Mitsunaga, and S. Nayar, "Efficient Space-Time Sampling with Pixel-wise Coded Exposure for High Speed Imaging." IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 36, Issue. 2, pp. 248-260, 2013. DOI: https://doi.org/10.1109/TPAMI.2013.129 https://doi.org/10.1109/TPAMI.2013.129
  8. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. European Conference on Computer Vision (ECCV), pp. 25-36, 2004. DOI: https://doi.org/10.1007/978-3-540-24673-2_3 https://doi.org/10.1007/978-3-540-24673-2_3
  9. A. Bruhn and J. Weickert. Towards ultimate motion estimation: Combining highest accuracy with real-time performance. International Conference on Computer Vision (ICCV), pp. 749-755, 2005. DOI: https://doi.org/10.1109/iccv.2005.240 https://doi.org/10.1109/iccv.2005.240
  10. L. Xu, J. Jia, and Y. Matsushita, "Motion detail preserving optical flow estimation." IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 34, Issue. 9, pp. 1744-1757, 2012. DOI: https://doi.org/10.1109/TPAMI.2011.236 https://doi.org/10.1109/TPAMI.2011.236