1. Introduction

Accidental fires cause severe economic and ecological damage in addition to threatening people’s lives. To avoid the catastrophe of a fire, fire-detection techniques were developed which were mainly based on particle sampling [1], temperature sampling [2], in addition to the traditional ultraviolet [3] and infrared [4] fire detectors and sensors. Sensor-based smoke detection systems work effectively if they are located close to the smoke source. One of the critical limitations of these systems is their inability to identify and locate the source the smoke, especially if deployed in outdoor environments.

As alternative to sensors, image-based detection systems of indoor and outdoor fires have been developed as an effective alternative to detect smoke. The concept relies on the development of an algorithm which is able to automatically detect very early signs of smoke in captured images. One of the most challenging issues in such techniques is the recognition of smoke in outdoor environments. This is due to the difficulty of recognizing the visual characteristics of the smoke.

In this paper we present an alternative image-based smoke detection algorithm which works on recognizing fire smoke using colored digital images captured in outdoor environments. The proposed technique assumes that regular images of the area under surveillance are captured and immediately processed. These images will be analyzed and through a series of stages, including extracting the target, analyzing image colors and subtracting images blocks through an organized process. The significance of the proposed method lies behind its structure’s simplicity, smoke detection accuracy and performance. Compared to other techniques, our alternative method is found promising alternative as it offers high detection rates with non-complex design.

2. Literature Review

 The authors of [5] proposed a method which can detect smoke based on the concept of source separation. The method treats the captured fire images as a linear combination of smoke and background image pixels. In other words, the method aims to separate the smoke from the background of the captured images. The results published in this research show that the method is able to effectively detect smoke in outdoor environments. However, the performance of the proposed method is not examined, hence the method’s efficiency is questioned.

In [6], a method for image-based smoke detection using an image-based technique is presented.  The method relies on considering the static and dynamic features of the fire smoke. The dynamic features are disorder, growth, and frequent flicker, while the static features are self-similarity and wavelet energy. The first step is to detect the moving target in the image. This is done through the use of a median filter algorithm to retrain the noise. Consequently, a background subtraction technique based on adaptive a background update is applied. Upon detecting the moving target, the method aims to extract the target’s contour in every binary image of the target in order to extract a frequent-flicker feature in target boundaries. The disorder that may appear in the captured image can be measured based on the ratio of circumference to area for the segmented target region. The frequent-flicker feature is also extracted using a wavelet domain approach.  This can help in determining the temporal high-frequency activity in a pixel. The last step carried out in this method is the process of joining all extracted features through a joint feature vector into an artificial neural network.

The work in [7] is presented according to the fact that the boundary of the smoke region is not clear, which makes it difficult to extract the smoke region using fundamental image processing. To resolve this limitation, authors incorporated K-means algorithms to fix the initial point in the domain region. The image is completely black while all pixels are in the initial point and through an image segmentation process, the image of the smoke shape then develops self-similarity properties. The positional relationships of pixels extracted from a single brightness is used to finally apply the fractal encoding techniques used for detecting smoke images [8].

In [9] a method to improve the image quality of the smoke using fuzzy logic is presented. The smoke area is extracted using a Gaussian mixed model. A hyper parameter of a support vector machine is established for smoke recognition.

Different types of RGB-based models can be used for fire/smoke detection systems, including, YCbCr and YUV color models. The YCbCr color model is effective for fire and smoke detection using image processing techniques [10-12]. However, to minimize the chance of selecting moving objects, clustering algorithms such as the K-means algorithm are used.

A different type of technique is presented in [13]. The authors propose a real-time smoke classification method using texture analysis. This method works by background subtraction for moving objects using the Gaussian Mixture Model (GMM), which divides the image into background and foreground. Pixels that don’t match Gaussian distribution are considered as foreground pixels. The image is divided into blocks, where each block is composed of 16*16 pixels as shown in Figure 1.

Figure 1: Sub-blocking the captured image [13]

For each block, the number of foreground pixels is counted. The method considers that particular block as a candidate block if and only if it has more than 205 foreground pixels. Consequently, the texture features are calculated for every candidate block using Gray Level Co-occurrence Matrices (GLCM). The texture features are classified under three categories: energy, contrast and homogeneity. A back propagation Neural Network (NN) algorithm is used to discriminate the smoke features. The three texture features form the input to the NN algorithm. The resulting output is a binary value (either 0 or 1), where the value 1 represents the existence of smoke in the image, and the value 0 represent the absence of smoke, as illustrated in Figure 2.

Figure 2: Detected smoke in the captured image [13]

In [14], the proposed method is designed based on deep saliency network. It aims to specify the important objects in an image. The output of this method is a smoke saliency map that is generated through the combination of pixel-level and object-level convolutional neural networks. The results show the method achieves good performance compared to some other method in that particular types of detection.

An improved smoke detection approach based on frame movement is presented in [15]. The approach relies on analyzing the characteristics of smoke at its early stages. The core of the approach is based on converting the captured images (extracted from multiple frames) to their binary representations. The lightness pixels that are not within the scope of the method are then removed. Subsequently, the smoke is detected using gray and transparency features. According to the performance analysis, the approach achieves a detection rate of about 92%.

In [16], the authors present an alternative smoke detection method based on visual smoke characteristics, including movement, color, gray tones, etc. When a region with movement is detected, the pixels within that region are estimated and analyzed to specify possible smoke regions. For higher accuracy, the method uses local binary patterns to characterize each region. Evaluation results shows that this method achieve an average smoke detection accuracy rate of 98.84%.

The authors in [17] proposes an algorithm to detect smoke and flame for video obtained from a camera installed in open area. The algorithm utilizes the adaptive background subtraction, and apply the optical flow-based movement estimation to identify a motion. Wavelet analysis is also applied to achieve moving blobs classification. Experiments results shows that the algorithm achieves a detection rate of 87% for smoke.

Similarly, the method presented in [18] utilizes the appearance and motion information to extract robust information. Machine learning is applied to achieve accurate smoke detection. Experiments results shows that the algorithm achieves a detection rate of 84.08%.

Finally, a video-based smoke-detection method is proposed in [19]. The core of the method relies on analyzing the spatial and temporal characteristics of video sequences to identify smoke features from possible smoke regions. These features include: edge blurring, gradual energy changes, and gradual chromatic configuration changes. These features are combined using a support vector machine technique and a temporal-based alarm decision unit to enhance the detection rates. Experiments results shows that the algorithm achieves a detection rate of 83.05%.

3. Alternative Smoke Detection Algorithm

In this section we present our proposed real-time smoke detection algorithm. The core of the algorithm relies on two main components: a target extractor and a color analyzer. In addition, the algorithm is provided with a supportive component for controlling the sensitivity of the detection process according to the area of application. The overall structure of the proposed algorithm is presented in Figure 3.

Figure 3: The structure of our proposed smoke detection algorithm

Figure 4: The internal structure of the TE Component

According to the above structure, the received image from the area under surveillance is processed through the Target Extractor (TE). This process assumes that the system is initialized by a base image in order to extract the target blocks which might be a candidate for a smoke block. Note that sensitivity controller is designed to control the sensitivity of the smoke detection process. Increasing the level of sensitivity results in detecting smoke in earlier stages. However, over-increasing in the sensitivity of the controller might increase the false positive ratios. The internal structure of the TE component is illustrated in Figure 4.

Upon completing the target extraction stage, the received image is then masked to compare the overall intensity against the base image. This operation is carried out as part of the Color Analyzer (CA). The CA component is designed to separate the RGB blocks to examine the standard deviation of the three mean values of the extracted blocks, as shown in Figure 5.

Figure 5: The internal structure of the CA Component

Consequently, a curve-fitting process is applied over the block intensity of both base image and current image obtained from the area under surveillance. The decision on whether smoke is found in the image or not is based on the difference between the fitted curves of both base and current images.

Figure 6: The base image of the area under surveillance

Figure 7: The image of the smoke captured for the area under surveillance

4. Results and Discussion

Our proposed system is tested against a set of images. Initially, the base-image should be stored for future comparison against the received image from the area under surveillance. Accordingly, Figure 6 presents the base image used in this experiment. For testing purposes, a fire has been started and the smoke starts to rise. Figure 7 shows the current image captured by the installed camera in the area under surveillance.

At this stage, a mask is applied over both the base image and the newly captured image to study possible changes in the overall intensity of the image’s blocks. As shown in Figure 8, the masking process shows exactly which areas are candidate to represent smoke.

Figure 8: Masking the base-image (A) and the newly captured image (B)

Upon completing both the target extraction and color analysis, our proposed algorithm considers only the blocks which includes the black smoke. In the next stage. The curve fitting process is applied. Figure 9 shows the base image intensity before applying the curve fitting, where the figure shows the mean value of each block in the image.

Figure 9: The intensity of the base-image (A) and the current image (B) before applying curve fitting

Figure 10: The intensity of the base image (A) and the current image (B) before applying curve fitting

Consequently, the curve-fitting process is applied over the base and current images. This process aims to find the best fit to the series of data points in each image. Figure 10 presents the intensity of the base-image and the current image after curve fitting has taken place.

Based on the two fitted curves of both the base and current images, one can note that the fitted curve of the current image (with smoke blocks) is lower than the fitted curve of the base-image. At this point, the level of sensitivity plays its role in whether to consider a particular block as smoke or not. We have conducted our experiment over a selected set of original smoke and none smoke images found in [20] and illustrated in Figure 11.

Figure 11: Set of Smoke Images (A-F) and Set of None Smoke Images (G-I)

The experiment results over the set of images shown in Figure 11, reveals that our method is able to detect the smoke-spectrum in most of the images. The results of the detection process applied over smoke and none smoke images are presented in Figure 12 and 13, respectively.

Table 1: Analyzing detection rates over different levels of sensitivity

Image	Level of sensitivity	Decision	False Alarm?
A. Smoke	50%	Smoke Detected	No
A. Smoke	75%	Smoke Detected	No
A. Smoke	95%	Smoke Detected	No
B. Smoke	50%	No Smoke	Yes
B. Smoke	75%	Smoke Detected	No
B. Smoke	95%	Smoke Detected	No
C. Smoke	50%	Smoke Detected	No
C. Smoke	75%	Smoke Detected	No
C. Smoke	95%	Smoke Detected	No
G. NonSmoke	50%	No Smoke	No
G. NonSmoke	75%	No Smoke	No
G. NonSmoke	95%	Smoke Detected	Yes
H. NonSmoke	50%	No Smoke	No
H. NonSmoke	75%	No Smoke	Yes
H. NonSmoke	95%	Smoke Detected	Yes

Figure 12: Original Smoke Images before Masking (A-F) and Masked Smoke Images (MA-MF)

Figure 13: Original None Smoke Images Before Masking (G-I) and Masked None Smoke Images (MG-MI)

Note that our proposed method is able to detect the smoke accurately as illustrated in Figure 12. However, our algorithm has also detected some smoke patterns in non-smoke images. This is due to the value assigned to the sensitivity controller. As highlighted in Section 3, the sensitivity controller plays a pivot role in formulating the final decision on whether a particular block of pixels is recognized as smoke or not. The sensitivity controller works on curve-fitting process. Increasing the sensitivity level to some extent, lead to higher false alarm rates. Table 1 shows how our algorithm react over different levels of sensitivities.

Obviously, the ideal level of sensitivity is found to be at rate 75%. Increasing or decreasing this ratio to some extent, results in higher rates of false alarms. However, it is a trade-off process, as some applications may require higher levels of sensitivity to avoid massive lost.

We run our experiment over 200 images. the results show that the detection rate is 95.1%. For comparison purposes, we compared our results with existing methods published in [15-19]. Results presented in Table 2 show that our method outperforms most of these methods in term of smoke detection rates. This is due to low-cost operations used in our method.

Table 2: Comparing the smoke detection rates of our proposed method against existing methods

Method	Smoke Detection Rate
Method 1 [15]	92.60%
Method 2 [16]	98.84
Method 3 [17]	87.00%
Method 4 [18]	84.08%
Method 5 [19]	83.05%
Our Method	95.10%

In terms of performance, our method was tested on an Intel Core i7 processor of 2.8GHz, a memory RAM of 3 GB, and hard disk of 80 GB. The results show that the average processing time for an image of size 92.5KB takes about 98ms. This performance makes our method suitable for deployment in a wide range of applications and environments.

5. Conclusion

This research presents an early smoke detection method using image processing techniques. The method is based on target extraction, color analysis and curve fitting to delimit the candidate smoke aregion. The level of sensitivity plays a pivot role in determining whether an alarm of smoke should be activated. The experiments results show that the proposed method provides a good detection rate of smoke against several scenarios. Results show that our method provides outstanding detection results at the sensitivity level of 75%, where the alarm was activated on all images that includes smoke patterns. Similarly, the images without smoke has not activated the alarm in more than 97% of the cases. The main reason behind some false alarm cases is the distance between the recording device and the target area. However, the intensive analysis of our method over a set of 200 images reveals that, the detection rate is 95.10%. In term of processing time, our method is capable of processing an image of less than 100KB in less than 1 second, making it feasible candidate for many environments.

Acknowledgment

The authors would like to thank the Arab Open University, Saudi Arabia for supporting this research.

1.  Kaiser, L. What Is an Air Sampling Smoke Detection System? Retrieved May 11, 2019, from Mission Critical Fire Protection: https://www.orrprotection.com/mcfp/blog/air-sampling-smoke-detection-system, 2015.
2.  Shixing, L., Xinxin, L., Wei, Y., Changzheng, C., Kun, Y., Maoxiang, Y., Yongming, Z. Aspirating fire detection system with high sensitivity and multi-parameter. 2014 International Conference on Information Science, Electronics and Electrical Engineering. Sapporo, Japan: IEEE, 2014.
3.  Trumble, T. United States of America Patent No. US3982130A, 1976.
4.  Bosch, I., Serrano, A., & Vergara, L. Multisensor Network System for Wildfire Detection Using Infrared Image Processing. TheScientificWorldJournal, 1-10, 2013.
5.  Bchir, I. Image-based Smoke Detection Using Source Separation. Spectroscopy and Spectral Analysis, 39(3), 982-989, 2019.
6.  Xu, Z. Automatic Fire Smoke Detection Based on Image Visual Features. 2007 International Conference on Computational Intelligence and Security Workshops, (pp. 316-319). Heilongjiang, 2007.
7.  Fujiwara, N., Terada, K. Extraction of a smoke region using fractal cording. IEEE International Symposium on Communications and Information Technology. Sapporo: IEEE, 2004.
8.  M., Silva, Abu Baker, M., Petra, M. State of the art of smoke and fire detection using image processing. Int. J. Signal and Imaging Systems Engineering, 10(1), 22-30, 2017.
9.  Yuanbin, W. Smoke recognition based on machine vision. International Symposium on Computer, Consumer and Control (pp. 668-671). Xi’an: IEEE, 2016.
10.  Pu, R., Lee, S. Study of smoke detection by analysis of saturation in video. 2011 International Conference on Fluid Power and Mechatronics (pp. 169-172). Beijing: IEEE, 2011.
11.  Jinghong, L., Xiaohui, Z., Lu, W. The design and implementation of fire smoke detection system based on FPGA. 24th Chinese Control and Decision Conference (pp. 3919-3922). Taiyuan: IEEE, 2012.
12.  Hauser, M., Li, Y., Li, J., Ray, A. Real-time combustion state identification via image processing: a dynamic data-driven approach. American Control Conference (pp. 3316-3321). Boston: IEEE, 2016.
13.  Yu, C., Zhang, J., Fang, J. Texture Analysis of Smoke for Real-Time Fire Detection. Second International Workshop on Computer Science and Engineering (pp. 511-515). Qingdao: IEEE, 2009.
14.  Xu, G., Zhang, Y. Zhang, Q., Lin, G., Wang, Z., Jia, Y., Wang, J. Video Smoke Detection Based on Deep Saliency Network. Fire Safety Journal, 105, 277-285, 2019.
15.  Mutar, A., Dway, H. Smoke Detection Based On Image Processing by Using Grey and Transparency Features. Journal of Theoretical and Applied Information Technology, 96(21), 6995-7006, 2018.
16.  Mercado, J., Medina, K., Perez, G., Suarez, A., Meana, H., Orozco, A., Villalba, L. Early Fire Detection on Video Using LBP and Spread Ascending of Smoke, Sustainability, 11(3261), 1-16, 2019.
17.  Ye, S., Bai, Z., Chen, H., Bohush, R., Ablameyko, S. An effective algorithm to detect both smoke and flame using color and wavelet analysis. Pattern Recognition and Image Analysis. 27(1): p. 131-138, 2017.
18.  Avgerinakis, K., A. Briassouli, and I. Kompatsiaris. Smoke detection using temporal HOGHOF descriptors and energy color statistics from video. International Workshop on Multi-Sensor Systems and Networks for Fire Detection and Management, 2012.
19.  Lee, C., Lin, C., Hong, C., Su, M. Smoke detection using spatial and temporal analyses. International Journal of Innovative Computing, Information and Control. 8(7): p. 4749-4770, 2012.
20.  Zhang, Q. Research Webpage about Smoke Detection for Fire Alarm. Retrieved March 10, 2020, from State Key Laboratory of Fire Science: http://smoke.ustc.edu.cn/datasets.htm. 2011

(Click Here)

(Click Here)