Relevance of Automatic Number Plate Recognition Systems in Vehicle Theft Detection †

: Smart vehicle technologies have revolutionized human life in the current era. Smart vehicles, referred to as connected and autonomous vehicles (CAV) are equipped with advanced technologies that increase their safety and security. These technologies have the potential to transform various aspects of society in terms of transformation. This research paper presents an analysis of automatic number plate recognition (ANPR) systems and a comparison at each stage in the aspect of technologies and algorithms involving computer vision. The research paper compares algorithms used for number plate recognition at various ANPR stages. ANPR is also known as the automatic license plate recognition (ALPR) system in many countries. These ANPR systems are generally used in different applications like security surveillance, traffic management, and electric toll collection systems, including law enforcement, parking enforcement, etc. Several factors can destroy the performance of ANPR systems. These factors can lead to inaccuracies in plate recognition or cause the system to fail to identify license plates correctly. Some common factors that can undermine ANPR performance include poor image quality, nonstandard plates, weather conditions, vehicle speed, plate obstructions, lighting conditions, and hardware-based constraints. These challenges make ANPR an interesting area for research. In addition to enhancing the performance of ANPR, other technologies like RFID, and GPS can be used. The paper also focuses on the number plate recognition rate after applying different algorithms. This research aimed to improve the state of knowledge of ANPR, which includes various algorithms and ANPR steps analysis for number plate detection through citing relevant previous work.


Introduction
Vehicle theft is a major issue faced by the world these days.Intelligent transportation systems (ITS) is playing an important role in increasing the detection rate of vehicle thefts.The automatic number plate recognition system is already contributing as a part of ITS in the recognition of stolen vehicles.This does not need additional transponders to recognize registered number plates like radio frequency identification (RFID) systems.
ANPR cameras are very useful for not only capturing vehicle images but also helping to get additional vehicle information such as the vehicle's speed, direction, counting, and group of vehicle details.It is a cost-effective technology.These abilities make ANPR part of our lives and also promises to stay with us in the future [1].ANPR has also been adopted in many applications such as electric toll collection, traffic management systems, delivery tracking, user billing, queue length estimation, parking management systems, etc.The working process of mobile and fixed ANPR is shown in Figure 1.(a) Typical ANPR system diagram of a fixed ANPR system (right) and a mobile ANPR system (left) [2]; (b) license plate diversity in terms of color, clarity, font, size, style, etc. [2].
The general process of recognition is to capture the image as input and then the system extracts the number plate with the help of specific algorithms; the extracted image will then be processed for character segmentation.After segmentation, character recognition algorithms will be processed in the segmented output.This whole process makes use of different algorithms and can be performed in a few seconds.
Figure 1b shows the diversity of vehicle license plates in terms of style, color, quality, font, size, and other physical conditions.After capturing the image, the system has to localize the number plate and then segment the characters.If the extraction is not proper due to these diversities, the recognition rate of the overall system will be decreased.Although the ANPR is a challenging system due to its various stages, it is currently impossible to achieve 100% overall accuracy as each stage depends on the first step.Diversities such as different light conditions, shadows, unequal plate sizes, characters, variety of fonts, and background color means that it cannot produce adequate result accuracy in tough conditions [2].

Review of the Literature
ANPR technology is widely used in many regions because it helps to detect stolen vehicles and also helps to increase vehicle theft detection rates all over the world.ANPR is a combination of three stages.The recognition process is performed at the end of the third stage [1].The literature review considered various stages of ANPR technology.It also discussed the various algorithms used as well as the performance rate of implementation.To draw inferences from the proposed methodology, a comparative analysis was also conducted on all three stages based on different features and algorithms.The following ANPR stages have to be followed step by step.
Stage 1 Number Plate Extraction: it is the important and typical stage of the ANPR system.After capturing the vehicle image, the system extracts the number plate of the vehicle by using different algorithms.To distinguish between the license plate and other objects of the image, the algorithm should be reliable.There are many methods evaluated by researchers.Some of these are:

•
Edge Feature: several researchers used edge detection algorithms and filters to extract number plates.They used different datasets and different algorithms for the same such as vertical and horizontal edge histograms tested on 50 images with a 90% extraction rate [3].The extraction rate of the vertical edge detection algorithm (VEDA) tested on 50 images in different light conditions was 96% [4].Sobel, Gabor, and Canny edge detection filters [5], and also a combination of several algorithms were used by them.

•
Texture Feature: extracting number plates using texture features includes various algorithms such as Local Binary Pattern (LBP) which was tested on 110 images with an 89.7% accuracy rate [6].Histograms of Oriented Gradients (HOG) and techniques such as scan line technique had a 99.2% performance rate .The vector quantization The general process of recognition is to capture the image as input and then the system extracts the number plate with the help of specific algorithms; the extracted image will then be processed for character segmentation.After segmentation, character recognition algorithms will be processed in the segmented output.This whole process makes use of different algorithms and can be performed in a few seconds.
Figure 1b shows the diversity of vehicle license plates in terms of style, color, quality, font, size, and other physical conditions.After capturing the image, the system has to localize the number plate and then segment the characters.If the extraction is not proper due to these diversities, the recognition rate of the overall system will be decreased.Although the ANPR is a challenging system due to its various stages, it is currently impossible to achieve 100% overall accuracy as each stage depends on the first step.Diversities such as different light conditions, shadows, unequal plate sizes, characters, variety of fonts, and background color means that it cannot produce adequate result accuracy in tough conditions [2].

Review of the Literature
ANPR technology is widely used in many regions because it helps to detect stolen vehicles and also helps to increase vehicle theft detection rates all over the world.ANPR is a combination of three stages.The recognition process is performed at the end of the third stage [1].The literature review considered various stages of ANPR technology.It also discussed the various algorithms used as well as the performance rate of implementation.To draw inferences from the proposed methodology, a comparative analysis was also conducted on all three stages based on different features and algorithms.The following ANPR stages have to be followed step by step.
Stage 1 Number Plate Extraction: it is the important and typical stage of the ANPR system.After capturing the vehicle image, the system extracts the number plate of the vehicle by using different algorithms.To distinguish between the license plate and other objects of the image, the algorithm should be reliable.There are many methods evaluated by researchers.Some of these are:

•
Edge Feature: several researchers used edge detection algorithms and filters to extract number plates.They used different datasets and different algorithms for the same such as vertical and horizontal edge histograms tested on 50 images with a 90% extraction rate [3].The extraction rate of the vertical edge detection algorithm (VEDA) tested on 50 images in different light conditions was 96% [4].Sobel, Gabor, and Canny edge detection filters [5], and also a combination of several algorithms were used by them.

•
Character Feature: character feature-based techniques were also evaluated by researchers researching in the related field.These techniques are Hough transformation, scale-space analysis, adaboost classifiers, support vector machine (SVM), and scale invariant feature transformation (SIFT) [9].

•
Color Feature: the number plate can have a color diversity depending on the country.The researchers use several techniques such as the hue, lightness, and saturation (HLS) color model and 90% of the images were recognized correctly [10].The mean shift algorithm achieved 97.6% accuracy [11,12] and the fast mean shift method extracted the number plate from 400 images with an accuracy rate of 92.6% [13].In various proposals, preacquired images were used for extraction.Out of these studies, it was found that the LBP was the best algorithm to extract number plates with a 100% performance rate [20].
Stage 2 Character Segmentation: This stage takes place after the successful number plate extraction.This extracted number plate has been taken as input for this stage and the characters are segmented by the different features mentioned below:

•
Boundary Information: algorithms such as vertical edge detection, vertical histogram, closed curve techniques, morphological thickening, and morphological thinning on 1189 images with an 84.5% segmentation rate [24] were evaluated by researchers in their studies.

•
Connected Component Analysis: techniques like pixel connectivity were tested on 958 HD images with a 99.75% segmentation rate [21], connected component labeling and morphological method on 50 images with a 91% success rate [3], and a hybrid method of blob coloring and connected components with a 93.7% accuracy rate also proposed in previous studies.

•
Extracted Character Feature: an RGB color extractor was tested on 255 images with a 98.5% accuracy [25].YOLO models, YOLOv2, and Hough binarization methods were applied to 332 blurred images with 96.4% performance rate [26].Fast-YOLO model and a classification regression network (CR NET) was applied for segmentation by researchers.

•
Projection Method: vertical and horizontal pixel project methods were used for segmentation tested on 30,000 images with a 99.2% success rate [27] and profile projection methods were also used [28] on 560 images with a 95.4% rate.
Table 2 summarises different algorithms applied for these techniques to find out a better segmentation rate.The above techniques used for segmentation from past years.Out of these, CCA labeling with morphological operations performed with a 99.75% segmentation rate which was comparatively the best accuracy rate [21].
Stage 3 Character Recognition: recognition of segmented number plate characters is the final stage of image processing in the ANPR system.There are two major techniques used for optical character recognition (OCR) which are: • Template Matching: it is the simplest method of recognition.In this technique, segmented characters are compared with the existing template characters set.This process is performed by scanning the character's column vise, and the highest correspondence value is found as the best-matched character.The technique tested on 1200 blurred images in the dataset correctly recognized 90% of images [20] and when tested on 1300 images of size 640 * 480 pixels in the dataset, achieved a recognition rate of 92.5% [19].

•
Using Extracted Features: feature extraction is based on several algorithms and techniques such as SVM, hoteling transformation, etc. [31,32].
The techniques used for the same are use template matching, recognition using extracted features, etc. Table 3 shows the summary of these techniques.Out of these, OCR performed with a 99.50% accuracy rate [21].It is important to find out the best results at each stage to increase the overall recognition rate because the output of each stage is considered as the input of next stage.
All three stages perform an important role in number plate recognition.Whether there is a still image or a video, the technology can recognize number plates.After the recognition of vehicle number plates, the data can be used for postprocessing for vehicle theft detection.The retrieved dataset is sent to the connected server and compared with the existing vehicle theft database [35].This whole dataset can be used for several ITS in the future because ANPR stores all data in a central server.The review focuses on each stage and finds out the best algorithms working for every stage for further use in the proposed methodology.
Several researchers used HD images for number plate recognition.Tables 1-3 show results after applying different HD and blurred images in the dataset for recognition.
Different researchers combined different algorithms to improve the overall recognition rate.This combination of algorithms play an important role in vehicle theft detection.The tables illustrate fifteen diverse proposals for the ANPR system, out of which 64% of researchers employed HD-quality images and only 27% used images with various situations as input images shown in Figure 2.
Several researchers used HD images for number plate recognition.Tables 1-3 show results after applying different HD and blurred images in the dataset for recognition.Different researchers combined different algorithms to improve the overall recognition rate.This combination of algorithms play an important role in vehicle theft detection.The tables illustrate fifteen diverse proposals for the ANPR system, out of which 64% of researchers employed HD-quality images and only 27% used images with various situations as input images shown in Figure 2. In a real-time scenario, not all captured images can be HD if the vehicle is in motion or if the number plate is dirty or scrawled.Thus there is a need for a system that first clears and converts blurred images to HD and then all three ANPR stages will work step by step.However, in ANPR technology, the decision of using still image or video depends on a number of variables, applications, accuracy requirements, available resources, and budget; videos provide more contextual information and are more suitable for capturing moving vehicles.Hybrid strategies can incorporate the benefits of both for best results.

Experiment and Analysis
A controlled experiment was conducted for number plate detection using both HD and blurred image datasets.The successful implementation and evaluation of the system was tested using a 1920*1080 pixel HD dataset of 50 images at MATLAB device configuration.The experiment recognized and extracted number plates from various high-quality images with a high accuracy of 99%.Then, the performance of the system was rigorously evaluated on a separate test set composed of 30 different types of blurred image.The results underline the difficulties presented by unclear characters, touchy characters, and boundary problems, and an error was found.Through error analysis, the study was able to identify recurring failure patterns and gain knowledge of future scope and development.

Proposed Methodology
The existing studies show better results in the case of HD images, but in the case of blurred images, the recognition rate was too low.Therefore, in the proposed methodology, three new algorithms were introduced at the stage of preprocessing.The best extraction algorithm found was local binary pattern with edge information results with a 100% extraction rate [20], but it did not work with the best quality images.For segmentation, CCA labeling, and morphological operations, the results were 99.75%.Furthermore, based on the HD images dataset and for the last stage of ANPR, OCR algorithm results were 99.50% [21], as seen in Figure 6.Thus, in the proposed methodology, these algorithms will be applied with the addition of three image preprocessing algorithms to be applied to the blurred image dataset.
Figure 3 shows that the collected images were subjected to preprocessing.The combination of several techniques can help to enhance overall image quality because each algorithm focuses on different areas of image degradation.When a vehicle arrives at a toll plaza, the camera captures images of the vehicle's number plate, then the captured image In a real-time scenario, not all captured images can be HD if the vehicle is in motion or if the number plate is dirty or scrawled.Thus there is a need for a system that first clears and converts blurred images to HD and then all three ANPR stages will work step by step.However, in ANPR technology, the decision of using still image or video depends on a number of variables, applications, accuracy requirements, available resources, and budget; videos provide more contextual information and are more suitable for capturing moving vehicles.Hybrid strategies can incorporate the benefits of both for best results.

Experiment and Analysis
A controlled experiment was conducted for number plate detection using both HD and blurred image datasets.The successful implementation and evaluation of the system was tested using a 1920 × 1080 pixel HD dataset of 50 images at MATLAB device configuration.The experiment recognized and extracted number plates from various high-quality images with a high accuracy of 99%.Then, the performance of the system was rigorously evaluated on a separate test set composed of 30 different types of blurred image.The results underline the difficulties presented by unclear characters, touchy characters, and boundary problems, and an error was found.Through error analysis, the study was able to identify recurring failure patterns and gain knowledge of future scope and development.

Proposed Methodology
The existing studies show better results in the case of HD images, but in the case of blurred images, the recognition rate was too low.Therefore, in the proposed methodology, three new algorithms were introduced at the stage of preprocessing.The best extraction algorithm found was local binary pattern with edge information results with a 100% extraction rate [20], but it did not work with the best quality images.For segmentation, CCA labeling, and morphological operations, the results were 99.75%.Furthermore, based on the HD images dataset and for the last stage of ANPR, OCR algorithm results were 99.50% [21], as seen in Figure 6.Thus, in the proposed methodology, these algorithms will be applied with the addition of three image preprocessing algorithms to be applied to the blurred image dataset.
Figure 3 shows that the collected images were subjected to preprocessing.The combination of several techniques can help to enhance overall image quality because each algorithm focuses on different areas of image degradation.When a vehicle arrives at a toll plaza, the camera captures images of the vehicle's number plate, then the captured image is processed by the ANPR system and the preprocessing takes place and the system starts to clear the number plate with proposed algorithms.
is processed by the ANPR system and the preprocessing takes place and the system starts to clear the number plate with proposed algorithms.Starting with the motion blur removal, Gaussian blur removal or image sharpening algorithm, the image quality will be improved.These methods can help reduce noise, increase clarity, and improve overall image quality.Once the image is better, the number plate extraction method will be processed such as by LBP to gather texture data.Working with a clean and clear image will be beneficial as it can result in more accurate feature extraction.After extraction, the number plate characters will be segmented using CCA labeling and morphological operations and next and last stage of ANPR will be OCR to recognize the number plate.In the preprocessing step, we will implement the following algorithms systematically to improve image quality.
These three methods are often used for the following reasons: Motion Blur Removal: in the process of capturing vehicle images or video, motion blur is a common problem caused by the movement of the camera or vehicle about the seen.The sharpness and details that were lost due to this blurring effect can be recovered by using the motion blur.This method is extremely relevant when working with photos and videos that have a noticeable blurring in some directions.The images are restored after enhancing image quality in the database to their original sharpness.The reviewed research proposed a system to restore these sharp images by combining a pair of noisy/blurred images taken quickly to form a clear image.Deblurr RNN and Deblur-Merger, two neural network topologies, have been introduced to manipulate pairs of images sequentially and simultaneously [36].
Gaussian Blur Removal: gaussian blur can be caused by many things, including atmospheric effects and lens imperfections.Gaussian blurring reduction procedures can be used to improve small details and textures that may have been obscured by such blurring.It is useful to recover high-frequency features and edges in the image.Previous researchers found that this algorithm helped to filter images with heavy noise [37].
Image Sharpening: this algorithm is used to enhance the clarity of the images by its contrast at the edges, also useful for improving the sharpness of images.The algorithm can make edges appear crisper and more defined.According to the reviewed research, image sharpening using unsharp masking (UM) approaches supports moderate contrast detail enhancement.The contrast and brightness of the image are additionally enhanced via waveletbased UM, which also provides good value parameter percentage correction [38].
The combination of these three algorithms enables us to treat a wide range of image degradation problems, thereby improving image quality overall.Depending on the exact Starting with the motion blur removal, Gaussian blur removal or image sharpening algorithm, the image quality will be improved.These methods can help reduce noise, increase clarity, and improve overall image quality.Once the image is better, the number plate extraction method will be processed such as by LBP to gather texture data.Working with a clean and clear image will be beneficial as it can result in more accurate feature extraction.After extraction, the number plate characters will be segmented using CCA labeling and morphological operations and next and last stage of ANPR will be OCR to recognize the number plate.In the preprocessing step, we will implement the following algorithms systematically to improve image quality.
These three methods are often used for the following reasons: Motion Blur Removal: in the process of capturing vehicle images or video, motion blur is a common problem caused by the movement of the camera or vehicle about the seen.The sharpness and details that were lost due to this blurring effect can be recovered by using the motion blur.This method is extremely relevant when working with photos and videos that have a noticeable blurring in some directions.The images are restored after enhancing image quality in the database to their original sharpness.The reviewed research proposed a system to restore these sharp images by combining a pair of noisy/blurred images taken quickly to form a clear image.Deblurr RNN and DeblurMerger, two neural network topologies, have been introduced to manipulate pairs of images sequentially and simultaneously [36].
Gaussian Blur Removal: gaussian blur can be caused by many things, including atmospheric effects and lens imperfections.Gaussian blurring reduction procedures can be used to improve small details and textures that may have been obscured by such blurring.It is useful to recover high-frequency features and edges in the image.Previous researchers found that this algorithm helped to filter images with heavy noise [37].
Image Sharpening: this algorithm is used to enhance the clarity of the images by its contrast at the edges, also useful for improving the sharpness of images.The algorithm can make edges appear crisper and more defined.According to the reviewed research, image sharpening using unsharp masking (UM) approaches supports moderate contrast detail enhancement.The contrast and brightness of the image are additionally enhanced via wavelet-based UM, which also provides good value parameter percentage correction [38].
The combination of these three algorithms enables us to treat a wide range of image degradation problems, thereby improving image quality overall.Depending on the exact implementation, dataset, and evaluation criteria employed, specific statistics and benchmarks are subject to change.
After improving image quality, the following algorithms shown in Table 4 will be applied.Overall, in the proposed methodology, first, we take a dataset of blurred images and then apply image preprocessing algorithms for motion blur removal, then gaussian blur removal, and finally image sharpening to enhance image quality.Then, we apply the algorithms mentioned in Table 4, step by step, and get the result.The results of both the HD and blurred image datasets will be compared to improve the recognition rate of the number plates as mentioned in the preprocessing phase.This proposed methodology will enhance image quality and help to improve the overall recognition rate.
Postprocessing of the system includes identification and comparison of the captured image data with the already existing blacklisted database of stolen vehicle number plates.This blacklisted data has been provided and circulated by the regional transport office (RTO) after lodging the first information report (FIR) by the vehicle owner at the nearest police station [9].As resultant vehicle theft will be detected, the vehicle theft detection rate will be improved.

Conclusions and Future Scope
The paper has analyzed different algorithms used in number plate extraction, character segmentation, and character recognition in the automatic number plate recognition system.To a great degree, the success rate of the ANPR system highly depends on the quality of the captured vehicle number plate images.The study exhibits that the results of the ANPR system depend on the input images.In the process of picking out the best algorithms, local binary patterns with edge information for number plate extraction, CCA labeling, morphological operations for character segmentation, and OCR algorithms for character recognition were found to be the best algorithms at each stage, respectively.These algorithms are used to enhance the overall system recognition rate but with one precondition of the images being of HD quality.
Thus, the research further aimed to enhance results for blurred images or tampered images along with all of the best algorithms.For this, the research included three image quality-improving algorithms: 1. image sharpening, 2. motion blur removal and 3. gaussian blur removal at the stage of image preprocessing.The research is consistent with other ANPR research initiatives in several ways.(a) Addressing image challenges: previous work identified difficulties in identifying lower quality and poor images.(b) Leveraging enhancement techniques: to enhance image quality, the idea uses established methods from the field of computer vision.(c) Addressing the detection gap: the proposed method focuses on detection reduction for blurred images, aligning with ANPR research to enhance the overall performance of the system.
In short, the proposal advances previous ANPR work by tackling image quality issues with tried and true methods and improving ANPR accuracy.Overall, the result of the study proves the applicability of image quality-improving algorithms to improve the overall number plate recognition rate.The study presents proposals on the relevance of automatic number plate recognition systems in vehicle theft detection.The research promises enhanced results and the results will be encouraged for further research.For future research, there is a need to simulate and implement the results on blurred or tempered image datasets.The dataset can be collected from toll plazas as a primary dataset for future research.The results could be compared and the efficacy of the proposed algorithms could be found.Moreover, the proposed algorithms could be tested on real-time images rather than prestored datasets.The results can be improved by applying new algorithms at different ANPR stages.The device configuration can also be changed accordingly.

Figure 2 .
Figure 2. Literature coverage based on different types of image.

Figure 2 .
Figure 2. Literature coverage based on different types of image.

Figure 3 .
Figure 3. Process of proposed automatic number plate recognition system.

Figure 3 .
Figure 3. Process of proposed automatic number plate recognition system.

Table 1 .
Diverse proposals that apply on number plate extraction features.

Table 2 .
Diverse proposals that apply on number plate segmentation features.

Table 3 .
Diverse Proposals that Apply to Number Plate Recognition Feature.

Table 4 .
Performance summary of ANPR system technique (HD images dataset).