Optimized Multi-focus Image Fusion Using Genetic Algorithm

Optical imaging systems or cameras have a convex lens with limited depth of field. So, multi-focus images are obtained when such systems are used to capture an image of a particular scene. These images are fused to get all-in-focus image. This paper proposes a new simpler method of multi-focus image fusion. Instead of decomposing input image into blocks in pyramid style, the proposed algorithm considers the complete image for edge detection. The proposed algorithm uses the genetic algorithm (GA) to find out the optimum weights from extracted edges and then fuses the images with the fusion rule based on optimized weights. Experimental results show that this superimposition method performs well; consumes less computation time and thus proves to be suitable for hardware implementation.


Introduction
Multi-view information of the scene is obtained from different images captured using different angles and variable focal length of the lens of a camera. In practice, all cameras consist of convex lens and have limited depth of a field. Hence while capturing an image using such camera; certain objects appear sharp whereas others appear blurred depending upon the distance from the lens. Thus multi-focus images with images different objects in focus are obtained [1].
For human perception or machine vision, a well-focused image is preferred. This image can be obtained in optical or computational way. Optical way is to reduce the lens aperture so as to increase the focal length of a lens. But this increases the diffraction and degrades the image resolution. Also small amount of light enters the camera and dark image is produced. Deconvolution process when carried out on the image produces the sharp image in a computational way. But, this requires the knowledge of camera specifications and also results of this method are affected by the presence of noise [2]. So, the way out is to combine two or more multi-focus images together.This is multi-focus image fusion.
A single composite image is constructed by integrating the infocus portions of each image. Such fused images are also useful in biomedical imaging, target identification, microscopic imaging, military operations, machine vision, and object recognition and so on.
A large number of image fusion algorithms have been invented so far. These methods work in spatial domain and transform domain.
Spatial domain methods mainly work upon the image blocks. Sharper image blocks from multi-focus images of the same scene are selected based upon the sharpness measure such as EOG (Energy of Gradient) [3] and spatial frequency [4] and sometimes using focus detection algorithm [5][6][7][8][9]. The main difficulty of these methods is to select the block of optimum size, on which the quality of fused image depends. Too small block gets affected by noise and causes incorrect selection. Too large block size causes sharp and blur pixels to get into the same block which leads to blocking artifacts. Hence the quality of the fused image degrades. In [10], authors have used genetic algorithm to optimize size of the block to obtain better fused image. Proposed fusion techniques apply edge features of an entire image to the genetic algorithm.
In transform based methods, images are decomposed into constituent low resolution images using multi-scale and multiresolution transforms, due to which computational complexity increases. To reduce this computational complexity, a hybrid fusion technique of DWT and GA is proposed in [11]. This ASTESJ ISSN: 2415-6698 method is used to fuse thermal and visual satellite images and outperforms others. Generally, transform based methods consist of three steps viz. decomposition, coefficients fusion and reconstruction. These methods use Laplacian pyramid [12], DCT [3], Wavelet, Curvelet [13], Shearlet [14] and contourlet [15] etc. transforms for decomposition of images. Multi-scale decomposition effectively extracts the visually important information of the images such as lines and details; but it produces halo artifacts near the edges. Hence final fused image has to be sharpened so that edges which are vital for the interpretation of the scene become clear. So, authors in [16,17] suggest fusion methods which can preserve edges in the final fused image. Morphological toggle contrast operator [16] used to extract edge features which can be used for fusion; takes more time for feature extraction. Edge preserving method proposed in [17] uses decomposition based on weighted least squares filter for the fusion of multi-focus images using focus-measure based fusion rule. The method is fast and effective but the difficulty is to adaptively determine the number of decomposition levels in fusion process. In [18], a fusion method based on saliency detection is proposed. Initially, input images are decomposed into detail and approximation layers by using simple average filter. Then method constructs weight map depending upon focused regions detected using visual saliency. This method works faster in spite of decomposition and produces the fused image with better contrast.
In view of above analysis, authors propose the algorithm which can reduce the computational complexity due to decomposition with simple but fast fusion rule. The proposed novel technique uses three steps to fuse an image  It finds the edges using which features are extracted.  These features are used to calculate optimum weights with the help of genetic algorithm.  And finally images are fused with these weights using the fusion rule of superimposition.
Two fusion techniques are proposed here. They are edgesuperimposition, edge-GA-superimposition. This paper presents extended workoriginally presented in ICIS2016 [1]. The paper is organized as follows: Section II explains the proposed fusion techniques. Section III gives the objective evaluation parameters which are used to examine the performance of the proposed method. Section IV deals with the results and Section V is conclusion.

Edge Superimposition
Quality of the image depends upon the focus and sharpness of objects present in the image. So sharper images have more information than the blurred ones. Sharper images are obtained using fusion; but fusion makes edges blur. Thus to achieve an improvement in the sharpness of the fused images, this algorithm superimposes the edges on the input images. Sobel and Canny operators are used for edge detection. The edges are then averaged to form a new edge image E. This image is superimposed on the first input using weighted addition method.
The intermediate Image3 is superimposed on the second input image to form the final output image.
4 is taken to be unity as it is used for the intermediate image that is already superimposed.

Edge-GA-superimposition
Input images are converted into edge images using edge operators. Statistical features in the form of normalized central moments up to order 3 are extracted from each edge image. Total 16 moments are selected. Mean and standard deviation of edges (along x and y directions in case of Sobel) are also calculated. Finally, the feature vector containing the moments, mean and standard deviation is formed and its size is 1 X 20.
The genetic search [1,11] starts by generating a population of size 100. This population consists of randomly generated binary strings of length 10. Before evaluating the individuals, they are converted from binary to decimal equivalent such that every binary string lies in the range of 0 to 1.This conversion is given as: Where, Lower limit (Ll) =0, Upper limit (Ul) =1, n=Number of bits in the binary.The individuals are evaluated using mean squared error as the fitness function. Mathematically, this fitness function is given as: d is the dimension of the input vector which is 1 x 40. F int (i) is the intermediate fused image obtained by using the equation: Img 1 and Img 2 are the input images, W int1 is the weight individual being evaluated and: The value of the fitness function should be as small as possible. However, it can also be transformed to make the fitness function a maximization fitness using the function: The individuals having the highest fitness values are selected. Different methods exist for the selection process such as-Roulette wheel selection, Rank based selection, and Tournament selection etc. [1,11]. Here tournament based selection method with a tournament size of 2 is used. Crossover and mutation operations with probability 0.7 and 0.002 respectively are performed on the selected individuals in the next step. Uniform crossover and bitflip mutation is used. These modified individuals replace the existing population. The process is repeated for 25 generations. It is experimentally observed that this search is converging in 20-23 generations. After the last generation, the optimum candidate is selected; which serves as first weightW 1 .The second weight is obtained W 2 =1 − W 1 . (12) On obtaining the two weights, each is assigned to the input images. Final fused image is obtained as a result of the addition of these optimally weighted images as: The block diagram for the process is described in Figure 1:

Evaluation of the fused images:
The basic requirement is that the output images should contain all the valid and useful information present in the source images without introducing any form of distortion. This can be verified by comparing the resultant images visually, but, these methods, though very powerful, are subjective in nature. Hence, objective statistical parameters are usedfor the evaluation of fused images [15,[19][20][21][22][23][24]. These parameters are defined as below.
Where L = dynamic range of the pixel-values; k 1 = 0.01 and k 2 = 0.03 by default.

Entropy:
Entropy is used to give the amount of information contained in an image. Mathematically it is given as: P i is the probability of occurrence of a pixel value in the image, L is the number of intensity levels in the image. Increase in the value of entropy after fusion indicates that the information contained in the image has increased.

Mutual Information:
It is an indicator of the information obtained from the source images and the quantity that is conveyed by the fused image. MI = MI F,I1 + MI F,I2 Where MI F,I1 denotes the mutual information between the fused image and the first input image, MI F,I2 denotes the mutual information between the second input and the fused image, P F,I1 (f, i1) and P F,I2 (f, i2) are the joint histograms of fused image, input 1and fused image, input 2 respectively, P F (f), P I1 (i1) and P I1 (i2) are the histograms of the fused image, input 1 and input 2 respectively.

Run Time:
It gives a measure of the time required to execute the algorithm in seconds.

Image Quality Index:
Mathematically [9], IQI = where x ̅, y ̅ are the means of x and y respectively, σ xy is the covariance of x and y, σ x 2 , σ y 2 are the variances of x and y respectively.

Experimental settings
All algorithms are coded using Python and implemented on Intel dual core, i5, 3.14GHz processor. Edges of input images are extracted using Sobel and Canny operators in both the fusion techniques. DWT-GA which uses HARR Wavelets for the decomposition is implemented for the reference [1,11]. These algorithms are tested on different types of images from a database as well as on real time images captured by a camera (canon 60D, Tamron-SP 90 Di VC-macro), focusing on different object in every image. Different evaluation parameters mentioned previously are used for analyasis of results.

Results
Quantitative evaluation for different methods is carried out. Table I and II compares results for different parameters for Database and real time images respectively. They represent the average value of the respective evaluation parameter when the proposed algorithm is applied to database images and 20 real time images. Fig. 2 and Fig. 3 show results for sample input images taken from database and real time image sets respectively. These results are obtained after applying the proposed method to these samples. It was observed that proposed methods give better results compared to those obtained by DWT_GA [11]. The analysis also shows that for database images Sobel operator gives better performance whereas for real time images Canny operator can be preferred.

Conclusion
The follwing Table III gives us the consolidated perfomance of proposed techniques which work well with respect to DWT method. From above table it can be concluded that proposed edge based techniques work better than DWT based fusion techinque [11]. Edge GA based superimpostion method gives better performance for real time images as the weights are calculated and optimized using image statistics as compared to the constant values taken in edge superimposition method. The added advantage is its simplicity as decomposition and reconstruction of images is not required.As a result the method cosumes less run time compared to DWT based GA [11]. This property along with reduced computational complexity makes it simpler for hardware implementation and paralleling processing. Subjective analysis shows that the contrast of the images fused by the proposed method can be further improved. For real time images Canny based Genetic algorthm gives better performance but the computational time required is large. Thus, this algorithm can be further extended to fuse real time images in a quick and efficient manner which can be used in variety of applications.