Journal of the Institute of Electronics Engineers of Korea CI (전자공학회논문지CI)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Real-Time Fixed Pattern Noise Suppression using Hardware Neural Networks in Infrared Images Based on DSP & FPGA

DSP & FPGA 기반의 적외선 영상에서 하드웨어 뉴럴 네트워크를 이용한 실시간 고정패턴잡음 제어

  • 박장한 (삼성탈렉스(주) 종합연구소) ;
  • 한정수 (삼성탈렉스(주) 종합연구소) ;
  • 천승우 (삼성탈렉스(주) 종합연구소)
  • Published : 2009.07.25
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose design of hardware based on a high speed digital signal processor (DSP) and a field programmable gate array (FPGA) for real-time suppression of fixed pattern noise (FPN) using hardware neural networks (HNN) in cooled infrared focal plane array (IRFPA) imaging system FPN appears a limited operation by temperature in observable images which applies to non-uniformity correction for infrared detector. These have very important problems because it happen serious problem for other applications as well as degradation for image quality in our system Signal processing architecture for our system operates reference gain and offset values using three tables for low, normal, and high temperatures. Proposed method creates virtual tables to separate for overlapping region in three offset tables. We also choose an optimum tenn of temperature which controls weighted values of HNN using mean values of pixels in three regions. This operates gain and offset tables for low, normal, and high temperatures from mean values of pixels and it recursively don't have to do an offset compensation in operation of our system Based on experimental results, proposed method showed improved quality of image which suppressed FPN by change of temperature distribution from an observational image in real-time system.

본 논문은 냉각형 적외선(infrared focal plane array; IRFPA) 영상시스템에서 하드웨어 뉴럴 네트워크를 이용한 실시간 고정패턴잡음 제어를 위해 고속 DSP & FPGA 기반의 H/W 설계 방법을 제안한다. 고정패턴잡음은 검출기의 불균일 보정처리후에도 관측영상의 온도분포 변화에 의해 발생한다. 이것은 열상 화질의 저하뿐만 아니라 다른 응용에도 문제되는 중요한 요소이다. 냉각형 적외선 영상시스템의 신호처리구조는 저온, 상온, 고온의 3개 테이블을 기준으로 이득(gain) 값과 편차(offset) 값을 연산한다. 제안된 방법은 3개 편차 테이블에서 각각 교차되는 영역을 세분화하여 가상의 테이블을 만들고, 입력 영상의 구분된 3개 영역에서 영상의 평균값으로 하드웨어 뉴럴 네트워크의 가중치 값을 조정하여 최적의 온도구간을 선정한다. 이와 같은 방법은 영상의 평균값으로부터 저온, 상온, 혹은 고온의 이득, 편차 테이블을 연산하고, 운용 중에 지속적으로 편차 보상을 적용하지 않아도 된다. 따라서 제안된 방법은 실시간 처리로 관측영상의 온도분포 변화에 의해 발생하는 고정패턴잡음을 제어하여 영상화질의 개선된 결과를 보였다.

Keywords

References

  1. C. Park, K. Bae, and J. Jung, 'Object recognition in infrared image sequences using scale invariant feature transform,' Proc. The International Society for Optical Engineering (SPIE), Int. Conf. Defense and Security Symposium (DSS 2008), Vol. 6968, no. 69681P, pp. 1-9, April 2008 https://doi.org/10.1117/12.777976
  2. H. Lee, S. Kim, D. Park, J. Kim, and C. Park, 'Robust method for detecting an infrared small moving target based on the facet-based model,' Proc. The International Society for Optical Engineering (SPIE), Int. Conf. Defense and Security Symposium (DSS 2008), Vol. 6969, no. 69690E, pp. 1-9, April 2008 https://doi.org/10.1117/12.777485
  3. A. Averbuch, G. Liron, and B. Bobrovsky, 'Scene based non-uniformity correction in thermal images using Kalman filter,' Image, Vision Computing, Vol. 25, no. 6, pp. 833-851, June 2007 https://doi.org/10.1016/j.imavis.2006.05.019
  4. T. Zhang and Y. Shi, 'Edge-directed adaptive nonuniformity correction for staring infrared focal plane arrays,' Image, Vision Computing, Vol. 25, no. 6, pp. 833-851, June 2007 https://doi.org/10.1016/j.imavis.2006.05.019
  5. C. Park, J. Han, and K. Bae, 'Suppression of fixed pattern noise for infrared image system,' Proc. The International Society for Optical Engineering (SPIE), Int. Conf. Defense and Security Symposium (DSS 2008), Vol. 6968, no. 69681Q, pp. 1-9, April 2008 https://doi.org/10.1117/12.777985
  6. I. Lim, 'Implementation for temporal noise identification using adaptive threshold of infrared imaging system,' Proc. The International Society for Optical Engineering (SPIE), Europe Security and Defence (ESD 2007), Vol. 6737, no. 673718, pp. 1-10, October 2007 https://doi.org/10.1117/12.738681
  7. S. Chun, Y. Hayakawa, and K. Nakajima 'Hardware neural network for a visual inspection system,' IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences 2008, Vol. E91-A, no. 4, pp. 935-942, April 2008 https://doi.org/10.1093/ietfec/e91-a.4.935
  8. F. Chen and S. Liu, 'A neural-network approach to recognize defect spatial pattern in semiconductor fabrication,' IEEE Trans. Semiconductor Manufacturing, Vol. 13, no. 3, pp. 366-373, August 2000 https://doi.org/10.1109/66.857947
  9. G. Acciani, G. Brunetti, and G. Fornarelli, 'Application of neural networks in optical inspection and classification of solder joints in surface mount technology,' IEEE Trans., Industrial Informatics, Vol. 2, no. 3, pp. 200-209, August. 2006 https://doi.org/10.1109/TII.2006.877265
  10. H. Hikawa, 'A digital hardware pulse-mode neuron with piecewise linear activation function,' IEEE Trans. Neural Networks, Vol. 4, no. 5, pp. 1028-1037, September 2003
  11. H. Zhou, R. Lai, S. Liu, B. Wang, Q. Li, 'A new real-time processing system for the IRFPA imaging signal based on DSP&FPGA,' Infrared Physics, Technology, Vol. 46, no. 4, pp. 277-281, 2005 https://doi.org/10.1016/j.infrared.2004.02.003
  12. S. Sato, K. Nemoto, S. Akimoto, M. Kinjo, and K. Nakajima, 'Implementation of a new neurochip using stochastic logic,' IEEE Trans. Neural Networks, Vol. 14, no. 5, pp. 1122-1127, September 2003 https://doi.org/10.1109/TNN.2003.816341