1. Introduction

The system of automatic license plate number detection and recognition, known as ANPR (automatic number-plate recognition), is considered a great application and an example of the computer vision technology. ANPR is widely being used in, but not limited to, radar cameras, car park management and traffic management [1]. In Jordan, ANPR is being used in recording traffic violations (e.g., over-speeding), however it has several problems. In particular, some of the ANPR steps: cropping of the license plate and optical character recognition (OCR) are still being done manually [2]. This might be due to the high cost of a fully customized commercial ANPR system for detecting and recognizing the Jordanian license plates. Therefore, the main goal of this paper is to propose a low cost ANPR system that is customized for automatically detecting and recognizing the numbers of the Jordanian license plates. We aim to reduce and avoid human labor, and the possible human errors. Having human involved ANPR steps might result in misidentifying a vehicle plate numbers and data, which can cause erroneous tickets.

The main contributions of this work are as follows:

• Propose a low cost fully automated and customized Jordanian ANPR system.
• Propose and utilize an optimized calculation methodology of the rectangularity metric [3] to efficiently detect the Jordanian license plates.
• Build a database of Jordanian vehicles to be used as a benchmark for evaluating the performance of the proposed system.
• Compare the proposed system to other ANPR systems with the help of the built database for the Jordanian vehicles.
• Incorporate the recognition of the vehicle type and make into the system.

The rest of the paper is organized as follows. Section 2 presents the related work. Section 3 discusses in details the proposed system. Section 4 presents the experimentation results. Section 5 provides a discussion about the proposed system and the obtained results. Section 6 concludes the paper and provides some future work directions.

2. Related Work

Generally speaking, any ANPR system consists of four main steps as listed below, which are also shown in Figure 1.

1. Capturing an image of a vehicle.
2. Detecting vehicle’s license plate.
3. Segmenting out the license plate objects or characters.
4. Recognizing the segmented license plate characters using OCR (Optical Character Recognition).

Figure 1: ANPR steps

Based on the ANPR steps above, in the literature, several works have been proposed to address and optimize each step. Additionally, there is a significant work in combining the ANPR system with detecting and recognizing the make of the vehicle. In what follows, some of the related work regarding the ANPR systems and the vehicle make recognition systems are briefly presented.

In [4] and [5], the authors proposed an ANPR system, where mean shift segmentation is used to identify candidate plate regions. Then, the authors trained and used a Mahalanobis linear classifier with respect to three features to accurately detect number plate region. Namely, they used rectangularity [3], which is defined by the authors as “a measurement that represents how well an object fits its minimum enclosing rectangle”, aspect ratio and edge density features.

In [6], the authors proposed an Iranian ANPR system, where edge detection and morphological operators are used to identify license plate candidate regions. The percentage of rectangularity and color filter of each candidate plate region are used to accurately detect the plate region. Multilayer Perception (MLP) neural network is finally used for recognizing the numbers of the detected plate.

In [7], the authors proposed an Indian ANPR system. License plate detection is based on using Sobel filter edge detector, morphological operations and connected component analysis. The segmentation and recognition of plate characters, which are in different Indian languages, are performed based on training a Support Vector Machine (SVM) classifier.

In [2], the authors proposed a Jordanian ANPR system that is based on SIFT features to detect Jordanian license plates and recognize them. Template matching technique is used to locate special marks in the license plate to segment out the license plate. Finally, OCR is used to recognize the characters or numbers from the segmented plate.

In [8], the authors surveyed several keypoint features under the feature-based automatic license plate detection techniques. They categorized the vehicle detection techniques into motion and appearance based, which included sophisticated probabilistic modeling and feature extraction. Also, the authors surveyed and discussed vehicle recognition utilizing vehicle attributes like license plate, color, type, and logo.

In [9], the authors applied a bag of features method to the car make and model recognition (VMMR). The sparse coding technique based on square mapped gradients (SMG) feature descriptors is used to learn a dictionary of codewords. The authors then apply supervised K-means or SVM with SMG to recognize the car make and model. Finally, the authors compared the performance of the two approaches.

In [10], the authors proposed an automatic license plate recognition system to trap traffic violations. For example, to trap offenders disobeying the traffic rules in an enclosed space (e.g., campus).

In [11], the authors proposed an ANPR-based vehicle tracking system, which monitors vehicle’s Euclidean distance from a visible calibration line. License plate detection was based on the fact that license plate has an inherent property of having greater transitions in pixel values compared to that of its surroundings.

In [12], the authors proposed a number plate detection and localization system based on segmentation and morphological operators to segment out license plate from the captured image. Specifically, the authors used histogram checking, and vertical and horizontal edge detection methods for the segmentation of the license plate. The experimentation dataset consisted of 350 license plates from different countries including Jordan. The reported efficiency rate from the proposed system is 93.43%.

In [13], the authors proposed a Jordanian ANPR system, where the license plate was detected and localized based on a multistage algorithm using the aspect ratio and the edges ratio. Back propagation neural network was used for recognizing the license plate characters (OCR). The average success rate from the proposed system is 90% of 240 high resolution images of size 1188 × 960 pixel.

Very recently, in [14], the authors proposed an ANPR system to recognize the Bangladeshi characters in vehicle license plates. Morphological operation and histogram analysis with a dynamic threshold of vertical and horizontal projection profiles were applied in order to detect plate area. Character recognition is carried out using Support Vector Machine (SVM) classifier, where extracted Histogram of Oriented Gradient (HOG) features were used as input. The system was applied to 630 images of Bangla license plates of different categories of vehicles and achieved 91% accuracy in the extraction of plate ROI.

After surveying the literature and to the best of our knowledge, the work of [2, 12, 13] is the only published work that examines the detection and recognition of the license plates of the Jordanian vehicles. Therefore, the work presented in this paper extends the literature regarding the Jordanian license plates detection and recognition. In fact, the proposed work improves the automatic Jordanian license plate detection and recognition proposed in [2] by using another feature, which considers both the appearance shape of the license (rectangularity and aspect ratio) and the license plate content for detecting the license plate. Furthermore, the recognition of vehicle logo and make was also incorporated. It is worth mentioning the notes concerning the work by authors in [12] and [13]:

• The processing time for detecting the license plate and recognizing the plate numbers in both [12] and [13] was not reported at all, whereas in the proposed work it is being reported.
• The dataset of [13] consisted of high resolution images with-

out presenting any sample images. In the proposed work many sample images are provided from the created and employed dataset, which mostly consisted of mid to low resolution images, but better than 320 × 240.

• Neither the work in [12] nor in [13] presented a fully comprehensive ANPR system, where both the license plate type (private, public, government, etc.) and the vehicle’s make are being recognized similar to what are being implemented in the proposed work.

Additionally, it is noteworthy to mention that the proposed work shares some similarity with the work presented in [4] and [5]. The proposed system utilizes the rectangularity and aspect ratio metrics in detecting the license plate (Jordanian plate in our case), where it achieved the best plate detection accuracy compared to using the rectangularity methodology proposed in [4] and [5]. This will be discussed in more details in the next section.

3. Proposed System

This section presents the details of the proposed Jordanian ANPR system. Taking into consideration the main ANPR steps presented in Figure 1, in the designed and implemented ANPR system, the following steps were considered as shown in Figure 2. The system was fully implemented in MATLAB R2018a [15], where the graphical user interface (GUI) is shown in Figure 3.

Figure 2: The proposed ANPR steps

Figure 3: The graphical user interface of the proposed Jordanian ANPR system

3.1. Input image

The very first step in the system is to capture a vehicle image. An off-the-shelf webcam and a mobile phone camera were used to capture a set of database images to evaluate the system’s performance. Additionally some of the images were obtained from the Jordanian Opensooq website [16]. In general, the captured images should be of good illumination conditions whenever possible, capture either the back or the front of the vehicle, and have a good resolution (e.g., better than 320 x 240). The database consisted of 100 images, which corresponds to different illumination conditions, different backgrounds, and different perspectives of the captured Jordanian vehicles (i.e., angle and distance between the camera and the vehicle location). Figure 4 shows 16 randomly selected input images chosen from the created dataset. Several preprocessing steps are then being performed on each input image that included histogram contrast enhancement.

Figure 4: 16 randomly selected input images chosen from the created dataset

3.2. Vehicle detection

The second step is to detect the vehicle from the input image. Faster Regional Convolution Neural Network (FRCNN) [17], which is an objecte detection built-in MATLAB function, was trained to detect and determine the vehicle and its location in the image. FRCNN was chosen since it is significantly faster in training and testing sessions over other detection methods (e.g., R-CNN). Additionally, it is considered as a highly successful framework for generic object detection [18].

The process of using FRCNN is done iteratively among different zooming versions or scales of the input image. This iterative procedure stops, when a vehicle is found or the input image is zoomed to the maximum. In this case (the latter), it may be that no vehicle is found and thus the ANPR system will give an error message indicating such a state and halts. Otherwise (i.e., once the vehicle is found), a blue bounding box around the detected vehicle is drawn. The detected vehicle is then cropped and converted to gray scale. Figure 5 shows an example of this step.

Figure 5: An example of a given input image and the detected cropped vehicle

3.3. License plate detection

The third step is to detect the license plate from the cropped gray scale vehicle image, which is one of the most important ANPR steps. Initially, the fast marching segmentation method [19] is executed to find the best license plate candidate regions. Precisely, this segmentation method will segment the vehicle image into several connected components or parts. The segmented growing parts are initially initialized based on a uniformly chosen seeds within the cropped vehicle image as shown in Figure 6. One of those parts must be the license plate. Figure 7 shows some of the segmented parts as it was very difficult to visualize all candidate segmented parts for all seeds in the same image; as the segmented parts may overlap and therefore the segments will not clearly be viewed. To accurately identify the correct license plate region or part, the following analysis is performed:

• First, the shape of the Jordanian license plate is rectangle with two styles: European and American as shown in Figures 8 and 9, respectively. The European style places the numbers in one row. The American style places the numbers in two rows.
• Next, rectangularity measurement is used [3]–[5] to first identify the segmented parts whose shape is best described by a rectangle, but of certain aspect ratio. This is the aspect ratio, which is computed based on the ration between the height and width of either the American or European styles of the Jordanian license plate. Figure 10 correctly shows identified and detected license plate part based on the rectangularity measurement metric.
• Consequently, a red bounding box is drawn around the part, which satisfies the above constrain or criterion, and crop it as shown in Figure 11.

It is important to emphasize that one of the main contributions in this paper is the methodology used in computing the rectangularity measurement, and thus that for detecting the license plate more efficiently and precisely. Specifically, the presented methodology is quite different and efficient compared to that described in [4], [5] as explained next.

Figure 6: Uniformly chosen seeds in the vehicle cropped image to initialize the segmentation process for the plate detection step.

Figure 7: Some samples of the segmented vehicle parts after the fast marching segmentation during the license plate detection step.

Figure 8: European-style Jordanian plate.

Figure 9: American-style Jordanian plate.

Figure 10: Correctly identified and detected license plate part based on the rectangularity measurement metric.

Figure 11: Cropped license plate after being rotated as to maximize the rectangularity calculation

3.3.1. Contribution in Calculating the Rectangularity Measurement

The methodology for computing the rectangularity in [4], [5] requires that:

• each candidate license plate region is to be treated as a rigid object,
• computing its area A₀ by counting the total number of pixels in the region,
• finding its minimum enclosed rectangle (MER) fitted to its boundary by iteratively rotating this object in one degree steps in the range [−15^◦,15^◦] to account for possible tilted angles of the captured vehicles,
• computing the area of MER of this object at each incremental rotation,
• identifying the angle at which the MER goes through the minimum value (A_MER), and
• finally computing the rectangularity R at this angle as given in 1:

However, the methodology does not require rotating each candidate plate region in a range of steps, which we believe is an important step to perform in order to efficiently detect the correct license plate with high confidence. However, performing this step iteratively and incrementally might be very time consuming and might not produce precise rectangularity calculation. Alternatively,the following steps are performed, which constitute one of the main contributions of this work:

• Each candidate license plate region or object is checked to make sure it has a convex shape as was proposed in [20]. In case it is not convex, it is converted to a convex shape so that spaces in the outwards lines of the object (if any) are connected and filled. This process uses a set of morphological operation (dilation, followed by filling, and lastly followed by erosion).
• Next, ellipse fitting is applied to each convex shaped object. This helps us determine the major and minor axis’s of the fitted ellipse of the object.
• The object is then rotated as to remove the rotation angle, which is the angle between the elongation axis (major axis for the fitted ellipse) and the X-axis. Indeed, this is the angle at which the MER goes through the minimum value (A_MER).
• Then the area A₀ of the rotated object (the total number of pixels in the convex shape) is computed.
• Finally, the rectangularity R using Equation 1 is calculated.

3.3.2. Rectangularity Analysis

The methodology presented in [4], [5] was compared with the proposed methodology. It is important to mention to the best of our knowledge that we could not find any work other than [4], [5] which used the rectangularity metric as a major step in detecting and localizing the license plate similar to what is proposed in this paper. And, since this step is really the most critical step and the heart of any ANPR system, we believe that using the methodology of [4] and [5] as one of the baseline comparisons is still a fair choice especially with the proposed enhancement given in the previous subsection, which we believe shows an advancement to the literature in this direction. In the comparison, the dataset of 100 images is used to compute the detection accuracy of correctly finding the license plate region and the average rectangularity calculation time. The results are presented in Table 1. The results indicate that the proposed rectangularity calculation is more accurate, precise and faster. In fact, the analysis showed that proposed rectnagularity measurement values were very close to one for the candidate license plate objects compared to around 0.7 based on [4], [5]. Additionally, it is worth noting that the rectangularity measurement calculation under different ∆θ’s ∈ [−15,…,15] as in [4], [5] might not be unique. The analysis shows that the same rectangularity value can be obtained under multiple different ∆θ’s, which can be problematic and result in misidentifying the correct license plate region.

Table 1: Rectangularity analysis

Rectangularity methodology	Average plate detection accuracy	Average rectangularity calculation time (sec)
[4], [5]	65%	0.2785
The        proposed methodology	95%	0.0315

3.4. Jordanian license plate Verification

Upon the detection of a possible license plate region, it remains to verify that it is indeed a Jordanian plate. To do so, the label part is examined. It is located to the left side of the Jordanian license plate as shown in Figure 12. It contains the English and Arabic characters of the word Jordan. Therefore, it was decided to use template matching technique based on computing the score of the cross correlation coefficients between two binary windows; (the binary templates of the Arabic characters of the word Jordan and the binary connected components objects extracted from the anticipated label area of the identified license plate candidate). Following are the performed template matching steps:

• converting the label area into the binary scale using Otsu’s method.
• segmenting out all possible characters or objects within the label area by performing connected component analysis on them.
• resizing these segmented objects to match the size of the stored binary templates of the Arabic characters of the word Jordan
• exhaustively computing the correlation coefficient between each object and the templates.
  • if the computed correlation coefficients are above heuristically computed threshold, then the detected license plate is declared to be a Jordanian one.
  • otherwise, it gives an error message that it could not find a Jordanian plate and halts.

3.5. Determine license plate type

The fifth step is to determine the type of the Jordanian vehicle. This information is color-coded (see Table 2) in the label area of the license plate. Thus, in the proposed ANPR system, we analyze the color of the detected license plate label area in the original input image. Consequently, based on the analyzed color (most frequently occurred color value) and the recognized code part from the next ANPR step (numbers in first line in American license plate and the digits before the dot in European license plate), we determine the type of the vehicle. Figure 13 shows some examples of the detected Jordanian license plate types from the ANPR system. Table 3 summarizes the types found in the created dataset.

Figure 12: Label image part.

Figure 13: Types of Jordanian license plate [21].

Figure 14: License plate area that excludes the label region

Table 2: Color coded Jordanian license plate types [22]

The two reserved digits indication: selected list		License       plate color indication
1 – 5	Government vehicles	Red
6	Aqaba free zone vehicles	Red
7	Diplomatic vehicles	Yellow
8-9	Temporary entrance vehicles	Yellow
10 – 21	Passenger vehicles	White
41 – 42	Light goods vehicles	White
44	Tractors	Red
46	Motorcycles and Scooters	White
50	Taxi	Green
56	Small buses	White
58	Coaches	White
71	Trailer	Yellow

Table 3: Summary of the license plate types found in the dataset

Type of License Plate	Number of License Plates
Private, White	96
Diplomatic, Yellow	2
Taxi, Green	1
Temporary Entrance, Yellow	1
Total	100

3.6. Recognition of license plate numbers (OCR)

The sixth step is to recognize the license plate numbers. First, all possible characters within the detected license plate binary image are extracted by performing connected component analysis within the plate region. Each extracted connected component or object is resized to match the size of the stored binary templates of the numeric plate numbers (from 0 to 9) that are graphically shown in Figure 15. The typeface used to generate such templates is known as Mandatory [23], which is the same typeface that is used for the United Kingdom vehicle registration plates.

Figure 15: The stored binary templates (inverted in order to best be viewed) of the numeric plate numbers that are desired to be recognized (from 0 to 9)

 

Figure 16: More binary templates of the numeric plate numbers that are desired to be recognized (from 0 to 9)

Finally, the correlation coefficient between each object and the templates is exhaustively computed to recognize the plate numbers. It is noteworthy to mention that, in order to increase the OCR recognition accuracy, the number of stored binary templates for the character set ‘0123456789’ was increased as shown in Figure 16. This resulted in having better matching of the segmented objects and thus high confidence in the associated recognition results. Additionally, the license plate area that excludes the label region (Figure 14) is the OCR region of interest because the segmentation of possible numbers or objects is performed in it. As stated above, the Jordanian license plate splits into two different styles: European and American. Thus, the performed connected component analysis considers the properties and characteristics (distribution of the numbers) of the two types. It also considers the following facts:

• The width and length of all numbers are the same.
• The distance between numbers is nearly the same, except the code part (numbers in first line in American license plate and the digits before the dot in European license plate), which determines the type of vehicles such as: 50: taxi, 36: private bus, etc.).
• All numbers in the license plate have the same color, which is black. Additionally, the background color for the license plate (except label part) is white.

A green bounding box is drawn around each recognized number in the license plate region as shown in Figure 17. After applying the OCR, the code of the license plate is computed by calculating the distance between all numbers. The code part determines the type of the vehicle as we mentioned before and listed in Table 2.

Figure 17: OCR results: remove background from the license plate, segment-out the possible numbers, recognize them, and draw green bounding box around each recognized number.

Table 4: Summary of the vehicle makes found in the dataset

Vehicle Make Index	Vehicle Make	Number of Vehicles
1	Mercedes	11
2	Toyota	10
3	Nisan	7
4	Hyundai	10
5	Mitsubishi	39
6	Opel	3
7	Audi	1
8	Ford	8
9	Mazda	1
10	Lexus	2
11	Daewoo	1
12	Jaguar	1
13	Jeep	1
14	Kia	2
15	Honda	1
16	Peugeot	1
17	Chrysler	1
Total number of Images		100

3.7. Recognizing the make of the vehicle

The seventh and final step of the proposed ANPR system is to determine the make of the vehicle. In this step, the make of the vehicle is recognized by localizing and recognizing the vehicle’s logo. Generally speaking, the fact that the vehicle’s logo is approximately located in the region above the detected license plate can be relied on, which can be a weak assumption. However, in this presented step, it is desirable to prove the concept of being able to add such feature (i.e., recognizing the make of the vehicle) easily. Thus, the vehicle’s logo in the segmented parts from the third step is searched for. The segmented parts are just above the detected license plate region. Once a logo was detected, a yellow bounding box is drawn around it. To help recognize those detected logos, several logo images from the Internet were collected. The images were specifically related to five auto makers MERCEDES, MITSUBISHI, KIA, NISSAN and TOYOTA. The list could be expanded easily. It should be mentioned that the dataset consisted of 17 vehicle makes or brands as summarized in Table 4, and the recognition focus was just put on five of them as indicated above. After that, using the collected logo dataset, a convolutional neural network (CNN) is trained with the help of MatConvNet MATLAB  toolbox to recognize the five makes of the vehicles as listed above. While testing this ANPR step on the input images dataset, it is assumed the case where none of the logo images of the vehicles would match the makes from which the CNN classifier is trained on as an ”UNKNOWN MAKE”. Figure

18 shows the recognized make for the vehicle that was shown in Figure 10.

Figure 18: The recognized make for the vehicle that was shown in Figure 10

To conclude this section, Figure 19 shows and summarizes the output of all stages of the proposed ANPR system. As it is clear from Figure 19:

• The detected vehicle is identified by a blue bounding box.
• The detected plate is identified by red bounding box.
• The detected logo of the vehicle is identified by yellow bounding box.
• The detected license plate is a Jordanian one and its type is private; since it has a white color and its code is recognized as “24”.
• The recognized license plate numbers are “12692”, where each number is bounded by a green bounding box.
• The recognized make of the vehicle is “Mitsubishi”.

4. Experimental results

This section presents the experimental results of the proposed ANPR system. A 100 images were tested as mentioned in the previous section. The average execution time for all tested images was found to be 8.0047 sec on average (ANPR steps 2-6) on a computer machine that has 16 GB of RAM and two quad core 2.5 GHz i7 processor. The system implementation was purely sequential and thus a single core was utilized. Table 5 presents the results (success rates or accuracies). The reported accuracy for each of the rows in Table 5 was computed for each step separately assuming correct results from previous step. As it can be seen from Table 5, the final license plate recognition accuracy from the OCR step was 94.68%. Consequently, the overall ANPR accuracy is 89% given the fact that the license plate detection step failed in 5 images, the license plate verification step failed in 1 image, and the OCR step failed in 6 images for a total of 11 images out of 100 (this is shown in the last row of Table 5). Finally, the accuracy for recognizing the make of the vehicles was 70.21% as given in the second row of Table 6 with an average execution time of 0.0096 sec. However the actual make accuracy was observed to be 86.84% as shown in the last row of Table 6. We should note that images without valid logos and unclear logos were manually excluded. Therefore, by excluding those images from the make evaluation dataset, the above accuracy was recomputed.

Table 5: The experimental results of the proposed ANPR system

ANPR Steps	Num. of input images	Num. of success images	Num. of            fail images	Accuracy	Average execution time
Step       2: Vehicle detection	100	100	0	100%	0.3394 sec
Step       3: License plate detection	100	95	5	95%	3.6327 sec
Step       4: License plate verification	95	94	1	98.95%	3.5924 sec
Step       5: License plate type	94	94	0	100%	0.00089 sec
Step       6: OCR	94	89	5	94.68%	0.4393 sec
ANPR system with       all Steps 2-6	100	89	11	89%	8.0047 sec

We have compared the proposed system with the following

ANPR systems: the ANPR system, which is available online at [24] and the ANPR system proposed in [2] (our previous work). For this comparison, we only used the 94 images out of the created dataset for which the license plates were successfully detected and verified as listed in Table 5. The result of this comparison is shown in Table 7, which shows that the proposed system achieved the best performance in terms of the recognition accuracy.

Figure 19: Summary of the output of all the steps of the proposed ANPR system.

Table 6: The make recognition results of the proposed ANPR system

ANPR Steps	Num.        of in- put images	Num. of success images	Num. of                fail images	Accuracy	Average execution time
Step 7: Make of the vehicle	94 (Successfully detected and verified license plate images)	66	28	70.21%	0.0096 sec
Step 7: Make of the vehicle	76 (Same as previous, but excluding the images that have un- clear/missing logos)	66	10	86.84%	0.0096 sec

Table 7: Comparison results with other systems

ANPR Systems	License Plate Recognition Accuracy
https://www.anpronline.net/ demo.html	56%
SIFT-based ANPR system [2]	75%
Our proposed system	94.68%

5. Discussion

As it can be inferred from Table 5, some of the steps need to be optimized either in terms of the average execution time or the accuracy of detection and recognition. For example, the average execution time of the license plate detection step was around 3.6327 sec and it was the largest among the rest of the steps. This is caused by the largest number of candidate segmented regions that were needed to check the license plate for; Rectangularity measurement and aspect ratio. Specifically, this is controlled by the number of the initialized seeds which is equal to 256 = 16 × 16 uniformly spaced grid as was shown in Figure 6. Despite that being said, it is important to highlight that our average calculation time of the rectangularity measurement per each candidate license plate part was better compared to the work of [4], [5] as was shown in Table 1 with a detection accuracy of 95%. Studying the factor of reducing the number of initialized seeds to reduce the license plate detection time is planned as a future work and how would that affect the detection accuracy.

Another observation from Table 5 is that the next highest average execution time was the license plate verification step. We believe that this is related to the way in which the verification process was performed as large number of small sized binary segmented windows were exhaustively compared against the binary templates of the English and Arabic characters of the word Jordan. Moreover, we carefully investigated the single failure case from this step, and it turned out to be caused by the bad resolution cropped license plate. The produced binary image of the cropped license plate is as shown in Figure 20.

Additionally, based on the results obtained from the OCR step, it was found that the main causes for the relatively low OCR recognition rate are related to one or more of the following factors:

• The numbers on some of license plates were not so much clear in the captured input images or may have bad illumination conditions; such as the example shown in Figure 21.
• Some of the numbers have certain painting defects that cause the OCR algorithm to not recognize them correctly such as the example shown in Figure 22 for the number ‘6’.

These kind of problems result in a bad segmentation of the plate characters and thus bad input to the OCR algorithm. Therefore, bad OCR recognition results.

Figure 20: The single case where the license plate verification step failed due to the bad resolution of the cropped license plate; the English and Arabic words of Jordan can not be found in the binary image.

Figure 21: A blurring problem in one of the numbers in the detected license plate (number 6) that caused the OCR step to fail

Figure 22: A painting defect problem in one of the numbers in the detected license plate (number 6) that caused the OCR step to fail

Moreover, the lowest recognition accuracy was for the step of recognizing the vehicle’s make. This was due to the assumption in localizing the logo of vehicle, which was assumed to always be found in the upper regions above the license plate. We believe that the main challenge was that the location of the logo above the license plate is not indeed consistent among all the vehicles, especially the tested ones (there were too much variations). Therefore, as was mentioned in Section 4 and after analyzing the captured input images, it was found that there were several images without any logo as can be seen in Figure 23, and in some of the images the logo was not clear due to the bad illumination conditions or due to the shadow effect as seen in Figure 24. It turned out that the number of images in the created dataset that possess these kind of problems was found to be 18. As a result of this observation and in order to test the performance of the trained make recognition subsystem, we excluded those 18 images from the make evaluation dataset (94 images). Consequently, the make recognition acucray was observed to be 86.84% as was shown in last row of Table 6.

For the final cost of the proposed ANPR system in terms of the hardware needed to run, it is clear that it only needs a computer machine and a camera to capture input images of the vehicles. Thus, considering the fact that cameras are now almost available everywhere (e.g., mobile phone cameras and surveillance cameras) and that much of the processing needed can be done on the cloud, the estimated cost of the proposed system could be as low as zero.

Figure 23: Four sample vehicle images, where the logo of vehicle that indicates its make is missing

Figure 24: Two sample vehicle images, where there is bad illumination conditions around the logo of each vehicle

Last but not least, although the proposed Jordanian ANPR system in this paper was shown to provide very good recognition results, however it may not work or produce correct results for all cases of the input images. Indeed it needs to be further analyzed under bad illumination conditions, blurriness in image, dirtiness or painting or physical defects problems in the numbers and characters on the license plate, and under the bad weather conditions (e.g., foggy and rainy conditions). About the speed performance of the proposed system, it is important to emphasis that the estimated time usually depends on the scene complexity of each input image, input image resolution, and the underlying speed of the running machine.

6. Conclusions and Future Work

The paper proposes a fully automated license plate recognition system targeting the Jordanian license plates as a case of study. The proposed system uses and optimizes the calculation of the rectangularity measurement to better identify the best license plate candidate region, which is considered one of the main contributions of the paper. The proposed Jordanian ANPR system was tested on a database that consisted of 100 Jordanian vehicles and compared it to two ANPR systems from the literature. Consequently, proposed system

achieves and reports the best performance among the compared systems in terms of the OCR recognition rate, which was 94.68%. We hope that the presented system will be adopted in the very near future in Jordan.

The presented system still needs to be optimized in different aspects as was discussed in Section 5, which is slated for future work. For example, it is desirable to study the effect of the selected number of seeds during the license plate detection step on license plate detection accuracy and the average detection time. Additionally, it is useful to analyze and further experiment the proposed system under bad illumination and weather conditions, blurriness in image, and under physical defects problems in the numbers and characters of the license plate. Moreover, examining better methods for recognizing the make of the vehicle and increasing the number of trained classes for the recognition of other makes are another area of future work.

Conflict of Interest

The authors declare no conflict of interest.

1. L. Keilthy, “ANPR system performance,” Parking trend international, 2008.
2. K. M. Ahmad Yousef, M. Al-Tabanjah, E. Hudaib, M. Ikrai, “SIFT based automatic number plate recognition,” in 2015 6th International Conference on Information and Communication Systems (ICICS), 124–129, IEEE, 2015, doi:10.1109/IACS.2015.7103214.
3. K. R. Castleman, Digital Image Processing, Prentice Hall, 1996.
4. W. Jia, H. Zhang, X. He, “Mean shift for accurate number plate detec- tion,” in Third International Conference on Information Technology and Applications, 2005. ICITA 2005., volume 1, 732–737, IEEE, 2005, doi: 10.1109/ICITA.2005.176.
5. W. Jia, H. Zhang, X. He, “Region-based license plate detection,” Jour- nal of Network and computer Applications, 30(4), 1324–1333, 2007, doi: 10.1016/j.jnca.2006.09.010.
6. S. Rastegar, R. Ghaderi, G. Ardeshipr, N. Asadi, “An intelligent control sys- tem using an efficient License Plate Location and Recognition Approach,” International Journal of Image Processing (IJIP) Volume (3), 3(5), 252–264, 2009.
7. M. Shidore, S. Narote, “Number plate recognition for indian vehicles,” IJC- SNS International Journal of Computer Science and Network Security, 11(2), 143–146, 2011.
8. R. A. S. Ma’moun Al-Smadi, Khairi Abdulrahimand, “Traffic Surveillance: A Review of Vision Based Vehicle Detection, Recognition and Tracking,” International Journal of Applied Engineering Research, 11(1), 713–726, 2016.
9. I. Kamal, K. Housni, M. Y. Hadi, “Car Make and Model Recognition using Sparse Coding and Bag of Features,” in Proceedings of the 2nd international Conference on Big Data, Cloud and Applications, 79, 1–5, ACM, 2017, doi: 10.1145/3090354.3090435.
10. P. Agarwal, K. Chopra, M. Kashif, V. Kumari, “Implementing ALPR for de- tection of traffic violations: a step towards sustainability,” Procedia Computer Science, 132, 738–743, 2018, doi:10.1016/j.procs.2018.05.085.
11. K. Indira, K. Mohan, T. Nikhilashwary, “Automatic License Plate Recognition,” in Recent Trends in Signal and Image Processing, 67–77, Springer, Singapore, 2019, doi:10.1007/978-981-10-8863-6 8.
12. A. A. Moustafa, M.-I. R. M. Jaradat, “A new approach for license plate detec- tion and localization: Between reality and applicability,” International Business Research, 8(11), 13, 2015, doi:10.5539/ibr.v8n11p13.
13. H. Alhaj Mustafa, S. Hassanin, M. Al-Yaman, “Automatic Jordanian License Plate Recognition System Using Multistage Detection,” in 2018 15th Inter- national Multi-Conference on Systems, Signals Devices (SSD), 1228–1233, IEEE, 2018, doi:10.1109/ssd.2018.8570408.
14. R. Islam, M. R. Islam, K. H. Talukder, “An efficient method for extraction and recognition of bangla characters from vehicle license plates,” Multimedia Tools and Applications, 20107–20132, 2020, doi:10.1007/s11042-020-08629-8.
15. The Mathworks, Inc., Natick, Massachusetts, MATLAB version 9.4.0.813654 (R2018a), 2018.
16. Available online: https://jo.opensooq.com/en/cars/cars-for-sale (accessed on 3 Jan 2020).
17. S. Ren, K. He, R. Girshick, J. Sun, “Faster R-CNN: Towards real-time ob- ject detection with region proposal networks,” IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 39(6), 1137 – 1149, 2017, doi: 10.1109/tpami.2016.2577031.
18. Q.-x. Zhang, G.-h. Lin, Y.-m. Zhang, G. Xu, J.-j. Wang, “Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images,”
19. J. A. Sethian, “A fast marching level set method for monotonically advancing fronts,” Proceedings of the National Academy of Sciences, 93(4), 1591–1595, 1996, doi:10.1073/pnas.93.4.1591.
20. P. L. Rosin, “Measuring shape: ellipticity, rectangularity, and triangular- ity,” Machine Vision and Applications, 14(3), 172–184, 2003, doi:10.1007/ s00138-002-0118-6.
21. Available online: http://www.plateshack.com/y2k/Jordan/ jordany2k.htm (accessed on 3 Jan 2020).
22. Available online: https://en.wikipedia.org/wiki/Vehicle_ registration_plates_of_Jordan (accessed on 3 Jan 2020).
23. Available online: https://en.wikipedia.org/wiki/Mandatory_ (typeface) (accessed on 3 Jan 2020).
24. “ANPRonline,” Available online: https://anpronline.net/index.html on 3 Jan 2020).

(Click Here)

(Click Here)