according to the present invention includes an infrared camera and a visible light camera, both viewing the same scene to be monitored. The visible light camera has a threshold means, for example, an optical filter, to attenuate the visible light input to the visible light camera to a level below which the visible light camera can not detect the scene. The output of the visible light camera indicates reflections of the sun light which are brighter than a predetermined threshold level. The output of the visible light camera is superposed over the temperature pattern of the scene measured with the infrared camera, so that the area having the reflection is deleted from the data of the temperature pattern. Thus processed temperature data is further processed with a conventional process so as to judge whether a rise in the temperature data is abnormal or not. The temperature monitoring system is therefore prevented from an erroneous operation caused by a reflection of the sun light in the scene.