1. Introduction

Image mosaicing or stitching is creating a panorama image by stitching or mosaicing many images that have overlapping points of the same view. A panorama is a large view of a scenery. It can be obtained by merging many pictures that have overlapping parts of the same scene. Image stitching is very popular in many applications such as construction of large view map, stitching arial images or satellite images and mosaicing radiography images in medical applications. In medical applications, it is needed to view the whole body x-ray images. The large view of x-ray images helps the surgeons to diagnose a disease for example long leg alignment operation, scoliosis patients etc. The traditional x-ray machine cannot produce large view x-ray image. Therefore, our proposed system can be applied in medical applications to view a panorama x-ray image.

Image stitching methods can be divided into two groups. They are direct method and feature based. Direct based method minimizes pixel to pixel mismatching, pixel based approach [1]. Feature based methods find distinctive features and then matched each other between two input images. The robust detectors can extract important characteristics (for example robust in translation, scale, noise and rotation). There has been great progress in medical images stitching system using features based approach. They are SIFT (Scale Invariant Feature Transform) [2], SURF (Speedup Robust Feature detector) [3], HARRIS detector [4] [5], Principal Component Analysis SIFT (PCA-SIFT) [6], and FAST (Features from Accelerated Segment Test). A good feature detector can be choose depending on the nature of the problem. Two main stages in feature based methods are the registration stage and the stitching stage.

The aim of this paper is to develop high quality image stitching system  with low processing time. The proposed system contains five main stages. They are preprocessing, feature detection and description, feature matching, Homography estimation and images stitching. This paper is an extension of work originally presented in 2018 International Conference on Intelligent Informatics and Biomedical Sciences [7].

The rest of this paper comprise of five parts. Part 2 presents the background of research work. Part 3 contains the briefly reviews of three features detection approaches. In Part 4, the proposed image stitching methodology is discussed. In Part 5, the results of proposed system are explained. Finally, Part 6 discusses the conclusion and the future works.

2. Background

Recently there has been done many researches works in image stitching using feature based approaches. Many researchers tried to develop the feature based methods that can detect reliable and repeatable features than traditional methods. But there still a great challenge in image stitching. There are many features detectors that were used in images stitching such as SIFT, SURF, Harris etc. Harris corners fail in image scaling and SIFT is the longest processing time [8].

According to the literatures, there are mainly two methods for image stitching. They are direct based approach and feature based method. Direct methods calculate the camera parameters by minimizing an error function based on the intensity difference of the overlapped region. Transformation matrix is calculated based on intensity difference. Direct based methods give accurate registration, but they fail in image scaling, illumination change and noise. Compared to feature based method, it requires a good initial estimation to achieve transformation matrix which is a major weak point of this method.

Feature based method detects different features, for example edges, points, corners, lines, or other shapes and calculates the relationship between these features. Compared to direct method, it is strong to light change, scale, affine transformation, noise, and orientation of the image. It takes care of locality around detected features to describe that feature as feature descriptor. In feature based method, all main features in an image pair are compared with that of every feature in other image by using the descriptors [9].

There are many feature detector methods exist that are used in medical images stitching system. Feature based approach is suitable for any movement of scene happened in image because of its robustness. This method is very faster, and it has the capability to identify a panorama image by detecting the neighbor correlation between two images. But, feature based techniques rely on accurate recognition of image features.

By using direct based approach, there had been done researched for medical images stitching. Among them, cross correlation was mostly applied because it can be developed using the FFT (Fast Fourier Transform) for stitching two images of the same size. But the weak point is high in costs [10].

There has been great progress in image stitching using feature based approaches. SIFT, SURF, and Harris detectors were mostly used [11]. Gong was presented an image stitching method. In feature extraction, SURF detector was used. The weak point is that it can’t detect correct features in image with noise [4] [5]. Singla presented images stitching using x-ray images by combining SIFT and SURF [12]. RANSAC ( Random Sample Consensus) algorithm was used to select the correct features points but high in computation complexity. Adel et al. [13] compared many feature based detectors. They showed that ORB was the fastest in processing time compared with other detectors.

Many researchers have been proposed the medical images stitching using feature based approach, but ORB method has not been applied yet. Therefore, this research work is the first work of biomedical images stitching using ORB feature based approach. The proposed system used ORB method in feature detection and description. The correct features points were chosen by using RANSAC algorithm. Homography matrix was estimated by applying the four feature pairs. As a final, the system mosaiced the two x-ray images together using Homography matrix. In blending stage, weight average method was used to eliminate the seam. The proposed system can be used in medical applications to help for surgeons to diagnose the diseases and to save time.

3. Feature Detection and Description

3.1.  SIFT

          SIFT (Scale Invariant Feature Transform) is an algorithm for local feature extraction and descriptor representation. The SIFT features are robust in image scale and rotation [3]. In SIFT algorithm, there are three major stages. These stages are key point or interest points detection, orientation assignment, and key point descriptor. Key point detection used difference of Gaussian function (DOG) to detect feature points which are invariant to scale and rotation. In orientation assignment stage, one or more orientations are assigned to each key point. In key point descriptor stage, a vector descriptor is developed for each key point.

3.2.  SURF

Bay et al. developed the SURF algorithm [1]. SIFT and SURF algorithms process a little different way in detecting features. SIFT creates an image pyramids and then filters each layer with Gaussians by increasing sigma values and taking the difference. SURF makes a stack without down sampling. By using integral images, SURF filters the stack using a box filter approximation of second order Gaussian partial derivatives. Integral images allow the computation of rectangular box filters in near constant time [6].

3.3.  Harris

Harris and Stephens  presented a corner detector, which is called Harris detector [14]. This detector is widely used to detect feature points and corners. To find the corners in the input image, Harris method takes look at the average intensity which is directional. The intensity change in the small specific area called window around an interested point. The point where the average intensity strongly high changes as compared to the previous one direction is called as a corner point.

4. Methodology

The proposed methodology of x-ray images stitching is shown in Figure 1. There are five main stages in our proposed system. They are preprocessing, feature extraction and description, feature matching, Homography estimation and images blending. Our proposed system accepted input x-ray images that have overlapped regions and recognized overlapped areas of the images by extracting the ORB features of those images and stitched them automatically to generate the final large view image. In our proposed system, there are two main components, namely overlapped area searching and stitching the images using the searched overlapped area. By using ORB features, the system can find the overlapped areas of the images efficiently. Moreover, the system can stitch the images to produce the seamless high resolution image with low processing time.

Figure 1. Proposed Images Stitching System Using ORB Feature Based Approach.

4.1.  Image Preprocessing

The preprocessing step is an essential step in image processing. First, the original DICOM format is converted into JPEG format to reduce the image size. We used 100 x-ray images with 50 pairs of overlapped images as the input images. It is required about 15-30% of overlapped regions to be stitched. The samples of the input x-ray images are illustrated in Figure 2.

4.2.   Features Extraction and Description

   Our proposed images stitching system used ORB feature based approach. ORB approach is a combination of Oriented FAST detector and Rotated BRIEF descriptor. Oriented FAST is used to detect feature points and Rotation BRIEF is used to generate descriptors. The speed of FAST and BRIEF are very fast, so ORB has advantages in speed. The good point of this algorithm is fast and having rotational invariance and reducing sensitivity to noise. Figure 3 and Figure 4 show the extracted ORB features of input x-ray images.

• Oriented FAST Detector

Both FAST algorithm and BRIEF algorithm do not have scale invariance. FAST detector can detect the features that are translation and rotation invariance, insensitive to noise and high reliability. But the feature points do not have information about direction. Therefore, ORB algorithm uses the Oriented FAST algorithm to detect the feature points which has direction information. Using intensity centroid method, it can calculate the direction of the corner and produce features with direction information.

From Equation 2, we can know the center of gravity or centroid.

The orientation of the vector from corner to center of gravity can be calculated by using Equation 3.

• Rotated BRIEF Descriptor

BRIEF produces binary descriptors by binary coding method. The binary descriptor is simple and storage space is smaller than SIFT and SURF.

Where p(x) is the pixel intensity at that point. The feature descriptor is defined as follows:

n BRIEF pairs (a1, b1), ……., (an, bn) can be written as a matrix S as follows:

A rotated version of  can be computed by using Equation 6.

Where,    is the test position for the binary pixel after rotation.

θ is the rotation angle,    is the rotation matrix of the main direction.

4.3.  Features Matching

        After detecting the ORB features, the next stage is features matching. In this step, hamming distance is applied to find the matched features. The results of features matching stage are shown in Figure 5 and Figure 6.

4.4.  Compute Homography Matrix

        After features matching, the next stage is Homography estimation. The RANSAC (Random Sample Consensus) algorithm is used to eliminate the wrong matched features and to compute the Homography matrix. The matching relationship of the images are found by using transformed Homography matrix.

4.4    RANSAC (Random Sample Consensus) Algorithm

RANSAC algorithm has three stages. In the first step, it chooses four random feature pairs. Second, it produces the Homography matrix for those selected feature pairs. In the third stage, it calculates an error measure pairs using Homography matrix.

• Homography Estimation from Point Correspondences

For two images that have a correspondence between 4 points, homography matrix can be expressed as below:

It is required 2 linear equations and 8 unknowns for each correspondence. Therefore, it is needed 8 linear equations for 4 correspondences. Two linear equations can be calculated by Equation 7 and 8.

In these equations, x_i^’ and y_i^’ are coordinates of changed stitched image.

Figure 2. The example input images that used in our proposed system.

4.5.  Images Blending

The final stage of images stitching is images blending. The aim of this step is to adjust the pixels intensity and to eliminate the visible seams. In this stage, the weighted average method is applied. Figure 7 and Figure 8 show the result images of our proposed system.

Figure 3. The result of features extraction from two knee x-ray images.

Figure 4. The result of features extraction from two leg x-ray images.

Figure 5. The result of features matching stage from the two knee x-ray images.

Figure 6. The result of features matching from the two leg x-ray images.

Figure 7. Final knee stitched image.

5. Results

The proposed system is the automatic images stitching system using medical x-ray images. Our proposed images stitching system is based on feature based approach. In feature detection and description, ORB method was used. The good point of ORB approach is rotation invariant, robust in noise and speed in processing time. We compared our proposed method with three different detectors, SIFT, SURF and Harris. According to the experiments, we can prove that our proposed method is the best in processing time and performance accuracy compared with three features detector methods. We examined the accuracy of our proposed method using 100 x-ray images. Our input samples consist of spine, lower limb and upper limb x-ray images. As the inputs images, our proposed system needs two overlapped x-ray images to be stitched. It is needed about 15-30% of overlapping regions to be stitched. The sample dataset was received from Department of Medicine, Aung Hospital, Lashio in Myanmar. Our proposed system can be used in medical applications such as scoliosis operations. The tested equipment to run for our proposed method is a computer with Core i7 CPU 2GHz and RAM 4GB. Table 1 and 2 describe the comparison of features extraction performance that tested with SIFT, SURF, Harris and ORB. Table 3 shows the comparison of quality measurement of the ORB method and other feature based image stitching methods using MSE and PSNR values. According to this table, images stitching using ORB method has lager PSNR value and smaller MSE value compared with other feature based methods. From experimental results, ORB can detect correct features points with lowest processing time. Therefore, ORB method is suitable for real time applications.

Figure 8. Final leg mosaiced image.

Table 1. The comparison of features extraction performance (SIFT, SURF, Harris and ORB) for Figure 2 (1) and (2).

Methods	Image Pixels	Features1	Features2	Time(s)	Matched features
SIFT	1295 x1224 1477 x1297	3295	1043	8.201892	211
SURF	1295 x1224 1477 x1297	459	873	0.682011	145
Harris	1295 x1224 1477 x1297	4321	2639	0.7356092	399
ORB	1295 x1224 1477 x1297	11	24	0.013491	4

Table 2. The comparison of features extraction performance (SIFT, SURF, Harris and ORB) for Figure 2 (3) and (4).

Methods	Image Pixels	Features1	Features2	Time(s)	Matched features
SIFT	1521 x1601 1521 x1275	1369	869	10.391036	304
SURF	1521 x1601 1521 x1275	287	172	0.876134	132
Harris	1521 x1601 1521 x1275	3922	2977	1.837210	457
ORB	1521 x1601 1521 x1275	56	43	0.018932	32

Table 3. Quality measurement of the resultant image that produced by ORB method and other feature based image mosaicing methods.

Methods	MSE	PSNR	Average Processing Time(s)
Image Stitching Using ORB	139.23	26.89	0.013491
Image Stitching Using SIFT	146.47	26.51	8.012932
Image Stitching Using SURF	186.75	25.91	0.65411
Image Stitching Using Harris	216.31	23.67	0.75314

6. Conclusion and Future Work

   In this paper, we proposed automatic images stitching system using feature based approach. In features detection and matching, ORB features were used. The proposed method was tested with 100 x-ray images with 50 overlapping pairs. The main goal of our proposed system is to develop a high quality images stitching system with low processing time. The contribution of our proposed system is that it is the first work of medical images stitching system using ORB approach. The proposed system was compared with other feature based images stitching algorithms, SIFT, SURF and Harris. The final image quality measured between the ORB algorithms and other feature based methods were computed using MSE and PSNR values. Experimental results show that the ORB algorithm can produce high quality seamless image mosaic of medical images with low processing time.

Our proposed method tends to be extended for 3D image stitching, 2D images stitching with moving objects and using hybrid of feature based with direct based approaches to get better results as a future work.

Conflict of Interest

The authors declare no conflict of interest.

Acknowledgment

The authors highly appreciate the Asian University Network (AUN/SEED-Net) for their financial support to contribute this research. Moreover, we also thank ‘Aung’ Hospital, Lashio, Myanmar, for the x-ray images used in this study.

1.  Niu Jing, Yang Fan and Shi Lingyi, “Improved method of automatic image stitching based on SURF,” First International Symposium on Future Information and Communication Technologies for Ubiquitous HealthCare (Ubi-HealthTech), Jinhua, pp. 1-5, 2013. https://doi.org/ 10.1109/Ubi-HealthTech.2013.6708059
2. Cansin Yildiz, “An Implementation on Recognizing Panoramas”, Department of computer engineering, Bilkent University, Ankara, Turkey, 2005. https://www.cs.bilkent.edu.tr/
3.  Matthew Brown and David Lowe, “Automatic Panoramic Image Stitching using Invariant Features”, International Journal of Computer Vision, Department of Computer Science, University of British Columbia, Vancouver, Canada, 2007. https://doi.org/10.1007/s11263-006-0002-3
4.  Jun Hui Gong, Jun Hua Zhang, Zhen Zhou An, Wei Wei Zhao and Hui-Min Liu, “An approach for X-ray image mosaicing based on Speeded-Up Robust Features,” International Conference on Wavelet Active Media Technology and Information Processing (ICWAMTIP), Chengdu,pp.432.435,2012.https://doi.org/10.1109/ICWAMTIP.2012.6413530
5.  R. Botan, K. F. Coco and K. S. Komati, “Implementation of an Image Stitching Algorithm to a Low-Cost Digital Microscope,” 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Niteroi, pp. 285-291, 2017. https://doi.org/ 10.1109/ICIHT.2017.7899142
6.  Luo Juan and Oubong Gwun, “A comparison of SIFT, PCA-SIFT and SURF,” 2009 International Journal of Image Processing (IJIP), pp. 143-152, 2009. https:// http://www.cscjournals.org/
7.  K. PyarWin and Y. Kitjaidure, “Biomedical Images Stitching using ORB Feature Based Approach,” 2018 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Bangkok, 2018, pp. 221-225. https://doi.org/ 10.1109/ICIIBMS.2018.8549931
8.  Chakrasali A. and Manjunatha P. “Automatic Image Mosaicing Using Discrete Cosine Transform”, Proceedings of International Conference on VLSI, Communication, Advanced Devices, Signals & Systems and Networking, Lecture Notes in Electrical Engineering, Vol 258, Springer, India, 2013. https://doi.org/10.1007/978-81-322-1524-0_35
9.  Samsudin, S., Adwan, S., Arof, H. et al. J, “Development of Automated Image Stitching System for Radiographic Images”, Journal of Digital Imaging, Volume 26, Issue 2, pp. 361-370, April 2013. https://doi.org/10.1007/s10278-012-9483-5
10.  M. Dawood, C. Cappelle, M. E. El Najjar, M. Khalil and D. Pomorski, “Harris, SIFT and SURF features comparison for vehicle localization based on virtual 3D model and camera,” 3rd International Conference on Image Processing Theory, Tools and Applications (IPTA), Istanbul, pp. 307-312, 2012. https://doi.org/ 10.1109/IPTA.2012.6469511
11.  Singla, Savita and Reecha Sharma. “Medical Image Stitching Using Hybrid of Sift & Surf Techniques.” International Journal of Advanced Research in Electronics and Communication Engineering (IJARECE) Volume 3, Issue 8, August 2014. https://www.citeseerx.ist.psu.edu/
12.  Ebtsam Adel, Mohammed Elmogy and Hazem Elbakry “Image Stitching System Based on ORB Feature-Based Technique and Compensation Blending”, International Journal of Advanced Computer Science and Applications, Vol. 6, No. 9, pp. 55-62, 2015. https://www. citeseerx.ist.psu.edu/
13.  S. Adwan and R. Majed, “A development of an augmented registration approach towards Scoliosis’s radiographic images stitching method,”. International Conference on Informatics, Health & Technology (ICIHT), Riyadh, pp. 1-6, 2017. https://doi.org/10.1109/ICIHT.2017.7899142
14. Chia Yen Chen, “Image Stitching Comparisons and New Techniques”, computer science department, University of Auckland, 1998. https://doi.org/10.1007/3-540-48375-6_73
15.  David Lowe, “Distinctive Image Features from Scale-Invariant Key points”, Computer Science Department, University of Columbia, International Journal of Computer Vision, Vol.60, No.2, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
16.  Yang Fan; He Yan; Deng Zhen Sheng and Yan Ang, “Improvement of automated image stitching system for DR X-ray images”, Computer in Biology and Medicine, Vol 71, pp. 108-114, 2016. https://doi.org/10.1016/j.compbiomed.2016.01.026
17.  B. Baheti, U. Baid and S. N. Talbar, “A novel approach for Automatic Image Stitching of spinal cord MRI images using SIFT,” International Conference on Pervasive Computing (ICPC), pp. 1-5, June 2015. https://doi.org/ 10.1109/PERVASIVE.2015.7087071
18.  G. Wang, Z. Zhai, B. Xu and Y. Cheng, “A parallel method for aerial image stitching using ORB feature points,” IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), pp. 769-773, 2017. https://doi.org/ 10.1109/ICIS.2017.7960096
19.  A. Pandey and U. C. Pati, “A novel technique for mosaicing of medical images”, Annual IEEE India Conference (INDICON), pp. 1-5, June 2014. https://doi.org/ 10.1109/INDICON.2014.7030465
20. E. Rublee, V. Rabaud, K. Konolige and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” International Conference on Computer Vision, Barcelona, pp. 2564-2571, 2011. https://doi.org/ 10.1109/ICCV.2011.6126544
21.  Hassaballah M., Abdelmgeid A.A. and Alshazly H.A. Image Feature Detectors and Descriptors. Studies in Computational Intelligence, SCI vol 630. Springer, 2016.

(Click Here)

(Click Here)