Automatic Stitching of Medical Images Using Feature Based Approach

,


Introduction
Image mosaicing or stitching is creating a panorama image by stitching or mosaicing many images that have overlapping points of the same view. A panorama is a large view of a scenery. It can be obtained by merging many pictures that have overlapping parts of the same scene. Image stitching is very popular in many applications such as construction of large view map, stitching arial images or satellite images and mosaicing radiography images in medical applications. In medical applications, it is needed to view the whole body x-ray images. The large view of x-ray images helps the surgeons to diagnose a disease for example long leg alignment operation, scoliosis patients etc. The traditional x-ray machine cannot produce large view x-ray image. Therefore, our proposed system can be applied in medical applications to view a panorama x-ray image.
Image stitching methods can be divided into two groups. They are direct method and feature based. Direct based method minimizes pixel to pixel mismatching, pixel based approach [1]. Feature based methods find distinctive features and then matched each other between two input images. The robust detectors can extract important characteristics (for example robust in translation, scale, noise and rotation). There has been great progress in medical images stitching system using features based approach. They are SIFT (Scale Invariant Feature Transform) [2], SURF (Speedup Robust Feature detector) [3], HARRIS detector [4] [5], Principal Component Analysis SIFT (PCA-SIFT) [6], and FAST (Features from Accelerated Segment Test). A good feature detector can be choose depending on the nature of the problem. Two main stages in feature based methods are the registration stage and the stitching stage.
The aim of this paper is to develop high quality image stitching system with low processing time. The proposed system contains five main stages. They are preprocessing, feature detection and ASTESJ ISSN: 2415-6698 description, feature matching, Homography estimation and images stitching. This paper is an extension of work originally presented in 2018 International Conference on Intelligent Informatics and Biomedical Sciences [7].
The rest of this paper comprise of five parts. Part 2 presents the background of research work. Part 3 contains the briefly reviews of three features detection approaches. In Part 4, the proposed image stitching methodology is discussed. In Part 5, the results of proposed system are explained. Finally, Part 6 discusses the conclusion and the future works.

Background
Recently there has been done many researches works in image stitching using feature based approaches. Many researchers tried to develop the feature based methods that can detect reliable and repeatable features than traditional methods. But there still a great challenge in image stitching. There are many features detectors that were used in images stitching such as SIFT, SURF, Harris etc. Harris corners fail in image scaling and SIFT is the longest processing time [8].
According to the literatures, there are mainly two methods for image stitching. They are direct based approach and feature based method. Direct methods calculate the camera parameters by minimizing an error function based on the intensity difference of the overlapped region. Transformation matrix is calculated based on intensity difference. Direct based methods give accurate registration, but they fail in image scaling, illumination change and noise. Compared to feature based method, it requires a good initial estimation to achieve transformation matrix which is a major weak point of this method.
Feature based method detects different features, for example edges, points, corners, lines, or other shapes and calculates the relationship between these features. Compared to direct method, it is strong to light change, scale, affine transformation, noise, and orientation of the image. It takes care of locality around detected features to describe that feature as feature descriptor. In feature based method, all main features in an image pair are compared with that of every feature in other image by using the descriptors [9].
There are many feature detector methods exist that are used in medical images stitching system. Feature based approach is suitable for any movement of scene happened in image because of its robustness. This method is very faster, and it has the capability to identify a panorama image by detecting the neighbor correlation between two images. But, feature based techniques rely on accurate recognition of image features.
By using direct based approach, there had been done researched for medical images stitching. Among them, cross correlation was mostly applied because it can be developed using the FFT (Fast Fourier Transform) for stitching two images of the same size. But the weak point is high in costs [10].
There has been great progress in image stitching using feature based approaches. SIFT, SURF, and Harris detectors were mostly used [11]. Gong was presented an image stitching method. In feature extraction, SURF detector was used. The weak point is that it can't detect correct features in image with noise [4] [5]. Singla presented images stitching using x-ray images by combining SIFT and SURF [12]. RANSAC ( Random Sample Consensus) algorithm was used to select the correct features points but high in computation complexity. Adel et al. [13] compared many feature based detectors. They showed that ORB was the fastest in processing time compared with other detectors.
Many researchers have been proposed the medical images stitching using feature based approach, but ORB method has not been applied yet. Therefore, this research work is the first work of biomedical images stitching using ORB feature based approach. The proposed system used ORB method in feature detection and description. The correct features points were chosen by using RANSAC algorithm. Homography matrix was estimated by applying the four feature pairs. As a final, the system mosaiced the two x-ray images together using Homography matrix. In blending stage, weight average method was used to eliminate the seam. The proposed system can be used in medical applications to help for surgeons to diagnose the diseases and to save time.

SIFT
SIFT (Scale Invariant Feature Transform) is an algorithm for local feature extraction and descriptor representation. The SIFT features are robust in image scale and rotation [3]. In SIFT algorithm, there are three major stages. These stages are key point or interest points detection, orientation assignment, and key point descriptor. Key point detection used difference of Gaussian function (DOG) to detect feature points which are invariant to scale and rotation. In orientation assignment stage, one or more orientations are assigned to each key point. In key point descriptor stage, a vector descriptor is developed for each key point.

SURF
Bay et al. developed the SURF algorithm [1]. SIFT and SURF algorithms process a little different way in detecting features. SIFT creates an image pyramids and then filters each layer with Gaussians by increasing sigma values and taking the difference. SURF makes a stack without down sampling. By using integral images, SURF filters the stack using a box filter approximation of second order Gaussian partial derivatives. Integral images allow the computation of rectangular box filters in near constant time [6].

Harris
Harris and Stephens presented a corner detector, which is called Harris detector [14]. This detector is widely used to detect feature points and corners. To find the corners in the input image, Harris method takes look at the average intensity which is directional. The intensity change in the small specific area called window around an interested point. The point where the average intensity strongly high changes as compared to the previous one direction is called as a corner point.

Methodology
The proposed methodology of x-ray images stitching is shown in Figure 1. There are five main stages in our proposed system. They are preprocessing, feature extraction and description, feature matching, Homography estimation and images blending. Our proposed system accepted input x-ray images that have overlapped regions and recognized overlapped areas of the images by extracting the ORB features of those images and stitched them automatically to generate the final large view image. In our proposed system, there are two main components, namely overlapped area searching and stitching the images using the searched overlapped area. By using ORB features, the system can find the overlapped areas of the images efficiently. Moreover, the system can stitch the images to produce the seamless high resolution image with low processing time.

Image Preprocessing
The preprocessing step is an essential step in image processing. First, the original DICOM format is converted into JPEG format to reduce the image size. We used 100 x-ray images with 50 pairs of overlapped images as the input images. It is required about 15-30% of overlapped regions to be stitched. The samples of the input x-ray images are illustrated in Figure 2.

Features Extraction and Description
Our proposed images stitching system used ORB feature based approach. ORB approach is a combination of Oriented FAST detector and Rotated BRIEF descriptor. Oriented FAST is used to detect feature points and Rotation BRIEF is used to generate descriptors. The speed of FAST and BRIEF are very fast, so ORB has advantages in speed. The good point of this algorithm is fast and having rotational invariance and reducing sensitivity to noise. Figure 3 and Figure 4 show the extracted ORB features of input x-ray images.

Oriented FAST Detector
Both FAST algorithm and BRIEF algorithm do not have scale invariance. FAST detector can detect the features that are translation and rotation invariance, insensitive to noise and high reliability. But the feature points do not have information about direction. Therefore, ORB algorithm uses the Oriented FAST algorithm to detect the feature points which has direction information. Using intensity centroid method, it can calculate the direction of the corner and produce features with direction information.
From Equation 2, we can know the center of gravity or centroid. The orientation of the vector from corner to center of gravity can be calculated by using Equation 3.

Rotated BRIEF Descriptor
BRIEF produces binary descriptors by binary coding method. The binary descriptor is simple and storage space is smaller than SIFT and SURF.
Where p(x) is the pixel intensity at that point. The feature descriptor is defined as follows: ( ) = ∑ 2 −1 ( ; , ) 1≤ ≤ n BRIEF pairs (a1, b1), ……., (an, bn) can be written as a matrix S as follows: A rotated version of can be computed by using Equation 6.
is the test position for the binary pixel after rotation. θ is the rotation angle, is the rotation matrix of the main direction.

Features Matching
After detecting the ORB features, the next stage is features matching. In this step, hamming distance is applied to find the matched features. The results of features matching stage are shown in Figure 5 and Figure 6.

Compute Homography Matrix
After features matching, the next stage is Homography estimation. algorithm is used to eliminate the wrong matched features and to compute the Homography matrix. The matching relationship of the images are found by using transformed Homography matrix.

RANSAC (Random Sample Consensus) Algorithm
RANSAC algorithm has three stages. In the first step, it chooses four random feature pairs. Second, it produces the Homography matrix for those selected feature pairs. In the third stage, it calculates an error measure pairs using Homography matrix.

Homography Estimation from Point Correspondences
For two images that have a correspondence between 4 points, homography matrix can be expressed as below: It is required 2 linear equations and 8 unknowns for each correspondence. Therefore, it is needed 8 linear equations for 4 correspondences. Two linear equations can be calculated by Equation 7 and 8.
In these equations, xi ' and yi ' are coordinates of changed stitched image. (1) (2) Figure 2. The example input images that used in our proposed system.

Images Blending
The final stage of images stitching is images blending. The aim of this step is to adjust the pixels intensity and to eliminate the visible seams. In this stage, the weighted average method is applied. Figure 7 and Figure 8 show the result images of our proposed system.

Results
The proposed system is the automatic images stitching system using medical x-ray images. Our proposed images stitching system is based on feature based approach. In feature detection and description, ORB method was used. The good point of ORB approach is rotation invariant, robust in noise and speed in processing time. We compared our proposed method with three different detectors, SIFT, SURF and Harris. According to the experiments, we can prove that our proposed method is the best in processing time and performance accuracy compared with three features detector methods. We examined the accuracy of our proposed method using 100 x-ray images. Our input samples consist of spine, lower limb and upper limb x-ray images. As the inputs images, our proposed system needs two overlapped x-ray images to be stitched. It is needed about 15-30% of overlapping regions to be stitched. The sample dataset was received from Department of Medicine, Aung Hospital, Lashio in Myanmar. Our proposed system can be used in medical applications such as scoliosis operations. The tested equipment to run for our proposed method is a computer with Core i7 CPU 2GHz and RAM 4GB. Table 1 and 2 describe the comparison of features extraction performance that tested with SIFT, SURF, Harris and ORB. Table  3 shows the comparison of quality measurement of the ORB method and other feature based image stitching methods using MSE and PSNR values. According to this table, images stitching using ORB method has lager PSNR value and smaller MSE value compared with other feature based methods. From experimental results, ORB can detect correct features points with lowest processing time. Therefore, ORB method is suitable for real time applications.

Conclusion and Future Work
In this paper, we proposed automatic images stitching system using feature based approach. In features detection and matching, ORB features were used. The proposed method was tested with 100 x-ray images with 50 overlapping pairs. The main goal of our proposed system is to develop a high quality images stitching system with low processing time. The contribution of our proposed system is that it is the first work of medical images stitching system using ORB approach. The proposed system was compared with other feature based images stitching algorithms, SIFT, SURF and Harris. The final image quality measured between the ORB algorithms and other feature based methods were computed using MSE and PSNR values. Experimental results show that the ORB algorithm can produce high quality seamless image mosaic of medical images with low processing time.
Our proposed method tends to be extended for 3D image stitching, 2D images stitching with moving objects and using hybrid of feature based with direct based approaches to get better results as a future work.