This paper presents a fire detection algorithm with a minimal false detection rate, intended for a thermal imaging surveillance environment, whose properties vary depending on temporal conditions of day or night and environmental changes. This algorithm was designed to minimize the false detection alarm rate while ensuring a high detection rate, as required in fire detection applications. It was necessary to reduce false fire detections due to non-flame elements occurring when existing fixed threshold-based fire detection methods were applied. To this end, adaptive flame thresholds that varied depending on the characteristics of input images, as well as the center of gravity of the heat-source and hot-source regions, were analyzed in an attempt to minimize such non-flame elements in the phase of selecting flame candidate blocks. Also, to remove any false detection elements caused by camera shaking, one of the most frequently raised issues at outdoor sites, preliminary decision thresholds were adaptively set to the motion pixel ratio of input images to maximize the accuracy of the preliminary decision. Finally, in addition to the preliminary decision results, the texture correlation and intensity of the flame candidate blocks were averaged for a specific period of time and tested for their conformity with the fire decision conditions before making the final decision. To verify the fire detection performance of the proposed algorithm, a total of ten test videos were subjected to computer simulation. As a result, the fire detection accuracy of the proposed algorithm was determined to be 94.24%, with minimum false detection, demonstrating its improved performance and practicality compared to previous fixed threshold-based algorithms.