Intelligent System for Vehicles Number Plate Detection and Recognition Using Convolutional Neural Networks
Abstract
:1. Introduction
- Localization of the number plate region: template matching algorithm is used for extracting the number plate region from the input image frame of the vehicle.
- Super resolution and segmentation techniques: the super resolution technique is used to get a clear number plate with good resolution and the bounding box method is used for segmenting each character of the number plate. The method segments the vehicle city, type, and number from the plate region.
- Feature extraction: the system used 700 number plate images for training by using CNN, and it provided 4096 features for each character to recognize correctly. The number plate images used in this investigation was collected from Bangladesh Road Transport Authority (https://service.brta.gov.bd/).
2. Related Works
3. Proposed Methodology
3.1. Pre-Processing
3.2. Localization of the Number Plate Region
3.3. Super Resolution Technique
Algorithm 1: Super Resolution of Plate Region |
Super_resolution For i = 1 to N For j = 1 to K SumFrame = SumFrame + Fk−1(Fk (H, Ok) − Ok); End loop; H = H – SumFrame; SR_Frame = H; Return SR_Frame; |
3.4. Segmentation of Character
Algorithm 2: Bounding Box Method of Plate Region |
For do For do Detect character as foreground object with a target bounding box; Form the target image region defined by the target bounding box; Normalization Region of Interest (ROI) with preservation of aspect ratio; Compute shaper description from normalized ROI, where is the underlying Riemannian manifold; End Collect the set of manifold points Compute final feature vector End Construct training set Train CNN classifier using with cross-validation |
4. Simulation Results and Discussions
- Step 1: the system captured a video of the vehicle. Then, the system extracted the vehicle frame from the video and localized the number plate from the vehicle image. The number plate images were converted to high-resolution images to perform accurate segmentation. For extracting the number plate, the template matching method was used.
- Step 2: for segmentation, the system used the bounding box method to segment each character. Each letter or word was mapped with a box value and extracted groups of characters. Figure 12 illustrates the segmented characters from the vehicle number plate images.
- Step 3: the system used CNN for extracting features and tested number plates on the VLPR vehicle dataset. In order to evaluate the experiment results, 700 vehicle images were appointed. The AlexNet model was employed for training the CNN. The system accomplished a maximum of 70 iterations for each input set. The iterations were confined when the minimum error rate was clarified by the user. The error rate for this system was 1.8%. After training, the CNN acquired 98.2% accuracy based on the validation set, and attained 98.1% accuracy based on the testing set.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Kocer, H.E.; Cevik, K.K. Artificial neural networks-based vehicle license plate recognition. Procedia Comput. Sci. 2011, 3, 1033–1037. [Google Scholar] [CrossRef][Green Version]
- Balaji, G.N.; Rajesh, D. Smart Vehicle Number Plate Detection System for Different Countries Using an Improved Segmentation Method. Imp. J. Interdiscip. Res. 2017, 3, 263–268. [Google Scholar]
- Annual Report, Bangladesh Road Transport Authority (BRTA). Available online: http://www.brta.gov.bd/ (accessed on 20 April 2020).
- Wu, C.; On, L.C.; Weng, C.H.; Kuan, T.S.; Ng, K. A Macao license plate recognition system Machine Learning and Cybernetics. In Proceedings of the 2005 International Conference on Machine Learning and Cybernetics, Guangzhou, China, 18–21 August 2005; IEEE: New York, NY, USA, 2005; 7, pp. 18–21. [Google Scholar] [CrossRef]
- Lopez, J.M.; Gonzalez, J.; Galindo, C.; Cabello, J. A Simple Method for Chinese License Plate Recognition Based on Support Vector Machine Communications. In Proceedings of the 2006 International Conference on Communications, Circuits and Systems, Guilin, China, 25–28 June 2006; IEEE: New York, NY, USA, 2006; 3, pp. 2141–2145. [Google Scholar] [CrossRef]
- Prabhakar, P.; Anupama, P. A novel design for vehicle license plate detection and recognition. In Proceedings of the Second International Conference on Current Trends in Engineering and Technology-ICCTET, Coimbatore, India, 8 July 2014; IEEE: New York, NY, USA, 2014. [Google Scholar] [CrossRef]
- Anagnostopoulos, C.N.E.; Anagnostopoulos, I.E.; Psoroulas, I.D.; Loumos, V.; Kayafas, E. License plate recognition from still images and video sequences: A survey. IEEE Trans. Intell. Transp. Syst. 2008, 9, 377–391. [Google Scholar] [CrossRef]
- Patel, C.; Shah, D. Automatic Number Plate Recognition System. Int. J. Comput. Appl. 2013, 69, 1–5. [Google Scholar]
- Du, S.; Ibrahim, M.; Shehata, M.; Badawy, W. Automatic license plate recognition (ALPR): A state-of the-art review. IEEE Trans. Circuits Syst. Video Technol. 2013, 23, 311–325. [Google Scholar] [CrossRef]
- Zhao, Z.; Yang, S.; Ma, X. Chinese License Plate Recognition Using a Convolutional Neural Network. In Proceedings of the 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application, Wuhan, China, 19–20 December 2008; IEEE: New York, NY, USA, 2008; pp. 27–30. [Google Scholar] [CrossRef]
- Telatar, Z.; Camasircioglu, E. Plate Detection and Recognition by using Color Information and ANN. In Proceedings of the 2007 IEEE 15th Signal Processing and Communications Applications, Eskisehir, Turkey, 11–13 June 2007; IEEE: New York, NY, USA, 2007. [Google Scholar] [CrossRef]
- Khan, N.Y.; Imran, A.S.; Ali, N. Distance and Color Invariant Automatic License Plate Recognition System. In Proceedings of the 2007 International Conference on Emerging Technologies, Islamabad, Pakistan, 12–13 November 2007; IEEE: New York, NY, USA, 2007; pp. 232–237. [Google Scholar] [CrossRef]
- Juntanasub, R.; Sureerattanan, N. Car license plate recognition through Hausdorff distance technique, Tools with Artificial Intelligence. In Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’05), Hong Kong, China, 14–16 November 2005; IEEE: New York, NY, USA, 2005. [Google Scholar] [CrossRef]
- Feng, Y.; Fan, Y. Character recognition using parallel BP neural network, International Conference on Language and Image Processing. In Proceedings of the 2008 International Conference on Audio, Language and Image Processing, Shanghai, China, 7–9 July 2008; IEEE: New York, NY, USA, 2008; pp. 1595–1599. [Google Scholar] [CrossRef]
- Patel, S.G. Vehicle License Plate Recognition Using Morphology and Neural Network. Int. J. Mach. Learn. Cybern. (IJCI) 2013, 2, 1–7. [Google Scholar] [CrossRef]
- Syed, Y.A.; Sarfraz, M. Color edge enhancement based fuzzy segmentation of license plates. In Proceedings of the Ninth International Conference on Information Visualisation (IV’05), London, UK, 6–8 July 2005; IEEE: New York, NY, USA, 2005; pp. 227–232. [Google Scholar] [CrossRef]
- Hidayah, M.R.; Akhlis, I.; Sugiharti, E. Recognition Number of the Vehicle Plate Using Otsu Method and K-Nearest Neighbour Classification. Sci. J. Inform. 2017, 4, 66–75. [Google Scholar] [CrossRef][Green Version]
- Liu, W.-C.; Lin, C.H. A hierarchical license plate recognition system using supervised K-means and Support Vector Machine. In Proceedings of the 2017 International Conference on Applied System Innovation (ICASI); Sapporo, Japan, 13–17 May 2017, IEEE: New York, NY, USA, 2017; pp. 1622–1625. [Google Scholar]
- Quiros, A.R.F.; Bedruz, R.A.; Uy, A.C.; Abad, A.; Bandala, A.; Dadios, E.P.; La Salle, D. A kNN-based approach for the machine vision of character recognition of license plate numbers. In Proceedings of the TENCON 2017—2017 IEEE Region 10 Conference, Penang, Malaysia, 5–8 November 2017; IEEE: New York, NY, USA, 2017; pp. 1081–1086. [Google Scholar]
- Thangallapally, S.K.; Maripeddi, R.; Banoth, V.K.; Naveen, C.; Satpute, V.R. E-Security System for Vehicle Number Tracking at Parking Lot (Application for VNIT Gate Security). In Proceedings of the 2018 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), Bhopal, India, 24–25 February 2018; IEEE: New York, NY, USA, 2018; pp. 1–4. [Google Scholar]
- Subhadhira, S.; Juithonglang, U.; Sakulkoo, P.; Horata, P. License plate recognition application using extreme learning machines. In Proceedings of the 2014 Third ICT International Student Project Conference (ICT-ISPC), Nakhon Pathom, Thailand, 26–27 March 2014; IEEE: New York, NY, USA, 2014; pp. 103–106. [Google Scholar]
- Singh, A.K.; Roy, S. ANPR Indian system using surveillance cameras. In Proceedings of the 2015 Eighth International Conference on Contemporary Computing (IC3), Noida, India, 20–22 August 2015; IEEE: New York, NY, USA, 2015; pp. 291–294. [Google Scholar]
- Sanchez, L.F. Automatic Number Plate Recognition System Using Machine Learning Techniques. Ph.D. Thesis, Cranfield University, Cranfield, UK, 2017. [Google Scholar]
- Panahi, R.; Gholampour, I. Accurate detection and recognition of dirty vehicle plate numbers for high-speed applications. IEEE Trans. Intell. Transp. Syst. 2016, 18, 767–779. [Google Scholar] [CrossRef]
- Rahman, M.S.; Mostakim, M.; Nasrin, M.S.; Alom, M.Z. Bangla License Plate Recognition Using Convolutional Neural Networks (CNN). In Proceedings of the 2019 22nd International Conference on Computer and Information Technology (ICCIT), Dhaka, Bangladesh, 18–20 December 2019; IEEE: New York, NY, USA, 2019; pp. 1–6. [Google Scholar]
- Leung, B.; Memik, S.O. Exploring super-resolution implementations across multiple platforms. EURASIP J. Adv. Signal Process. 2013, 1, 116. [Google Scholar] [CrossRef][Green Version]
- Lempitsky, V.; Kohli, P.; Rother, C.; Sharp, T. Image segmentation with a bounding box prior. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 29 September–2 October 2009; IEEE: New York, NY, USA, 2009; pp. 277–284. [Google Scholar] [CrossRef]
- Youcef, M.-A.H. Convolutional Neural Network for Image Classification with Implementation on Python Using PyTorch. 2019. Available online: https://mc.ai/convolutional-neural-network-for-image-classification-with-implementation-on-python-using-pytorch/ (accessed on 4 March 2020).
- Ghosh, A.K.; Sharma, S.K.D.; Islam, M.N.; Biswas SAkter, S. Automatic license plate recognition (ALPR) for Bangladeshi vehicles. Glob. J. Comput. Sci. Technol. 2011, 11, 1–6. [Google Scholar]
- Baten, R.A.; Omair, Z.; Sikder, U. Bangla license plate reader for metropolitan cities of Bangladesh using template matching. In Proceedings of the 8th International Conference on Electrical and Computer Engineering, Dhaka, Bangladesh, 20–22 December 2014; IEEE: New York, NY, USA, 2014; pp. 776–779. [Google Scholar] [CrossRef]
- Haque, M.R.; Hossain, S.; Roy, S.; Alam, N.; Islam, M.J. Line segmentation and orientation algorithm for automatic Bengali license plate localization and recognition. Int. J. Comput. Appl. 2016, 154, 21–28. [Google Scholar] [CrossRef]
- Khan, A.; Sohail, A.; Zahoora, U.; Qureshi, A.S. A survey of the recent architectures of deep convolutional neural networks. Artif. Intell. Rev. 2019, 53, 5455–5516. [Google Scholar] [CrossRef][Green Version]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar] [CrossRef][Green Version]
- Ramos-Llordén, G.; Vegas-Sánchez-Ferrero, G.; Martin-Fernandez, M.; Alberola-López, C.; Aja-Fernández, S. Anisotropic Diffusion Filter with Memory Based on Speckle Statistics for Ultrasound Images. IEEE Trans. Image Process. 2014, 24, 345–358. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Tsai, R. Multiframe Image Restoration and Registration. Adv. Comput. Vis. Image Process. 1984, 1, 317–339. [Google Scholar]
- Li, B.; Zeng, Z.Y.; Zhou, J.Z.; Dong, H.L. An algorithm for license plate recognition using radial basis function neural network. In Proceedings of the 2008 International Symposium on Computer Science and Computational Technology, Shanghai, China, 20–22 December 2008; IEEE: New York, NY, USA, 2008; pp. 569–572. [Google Scholar]
- Shan, B. Vehicle License Plate Recognition Based on Text-line Construction and Multilevel RBF Neural Network. J. Comput. Sci. 2011, 6, 246–253. [Google Scholar] [CrossRef]
- Mutholib, A.; Gunawan, T.S.; Kartiwi, M. Design and implementation of automatic number plate recognition on android platform. In Proceedings of the 2012 International Conference on Computer and Communication Engineering (ICCCE), Kuala Lumpur, Malaysia, 3–5 July 2012; IEEE: New York, NY, USA, 2012; pp. 540–543. [Google Scholar]
- Romadhon, R.K.; Ilham, M.; Munawar, N.I.; Tan, S.; Hedwig, R. Android-based license plate recognition using pre-trained neural network. Internet Work. Indones. J. 2012, 4, 15–18. [Google Scholar]
- Saif, N.; Ahmmed, N.; Pasha, S.; Shahrin MS, K.; Hasan, M.M.; Islam, S.; Jameel, A.S.M.M. Automatic License Plate Recognition System for Bangla License Plates using Convolutional Neural Network. In Proceedings of the TENCON 2019—2019 IEEE Region 10 Conference (TENCON), Kochi, India, 17–20 October 2019; IEEE: New York, NY, USA, 2019; pp. 925–930. [Google Scholar]
- Polishetty, R.; Roopaei, M.; Rad, P. A next-generation secure cloud-based deep learning license plate recognition for smart cities. In Proceedings of the 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, CA, USA, 18–20 December 2016; IEEE: New York, NY, USA, 2016; pp. 286–293. [Google Scholar]
- Chen, Y.S.; Lin, C.K.; Kan, Y.W. An advanced ICTVSS model for real-time vehicle traffic applications. Sensors 2019, 19, 4134. [Google Scholar] [CrossRef] [PubMed][Green Version]
Bangla letter | অ | ই | উ | এ | ক | খ | গ | ঘ | ঙ | চ | ছ | জ | ঝ | ত | থ | ড |
English letter | a | i | u | e | ka | kha | ga | gha | na | ca | cha | ja | jha | ta | tha | da |
Bangla letter | ঢ | ট | ঠ | দ | ধ | ন | প | ফ | ব | ভ | ম | য | র | ল | শ | স |
English letter | dha | ta | tha | da | dha | na | pa | pha | ba | bha | ma | ya | ra | la | sha | sa |
Bangla letter | ০ | ১ | ২ | ৩ | ৪ | ৫ | ৬ | ৭ | ৮ | ৯ | - | |||||
English number | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | - |
Reference | Detection | Segmentation | Recognition |
---|---|---|---|
Proposed system | For extracting plate region, bounding box method was used Super Resolution techniques were used to get clear images | Template matching | CNN |
[29] | Sobel edge detection with additional morphological operations | Line segmentation, word segmentation based on area filtering | Feed forward neural network |
[30] | Connected component technique | Template matching | |
[31] | Sobel edge detector | Line segment orientation (LSO) algorithm | Template matching |
[25] | CNN | CNN | CNN |
Reference | Sample Size | Localization | Accuracy | Processing Time |
---|---|---|---|---|
Proposed system | Training: 500 Testing: 200 | 100% | 98.2% | 111 milliseconds for the whole process |
[29] | Testing: 300 | 84% | 80% | - |
[30] | Testing: 120 | - | - | 1.3 s |
[31] | Testing: 119 | 95.8% | 84.87% | - |
[25] | Training: 450Testing: 50 | 88.67% | - | - |
Object | Parameter | Value |
---|---|---|
Letters | Recognition Rate (%) Recognition Times (ms) | 86.5 48.6 |
Numbers | Recognition Rate (%) Recognition Times (ms) | 97.8 48.9 |
Characters (Letters & Numbers) | Recognition Rate (%) Recognition Times (ms) | 90.9 52.3 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alam, N.-A.-; Ahsan, M.; Based, M.A.; Haider, J. Intelligent System for Vehicles Number Plate Detection and Recognition Using Convolutional Neural Networks. Technologies 2021, 9, 9. https://doi.org/10.3390/technologies9010009
Alam N-A-, Ahsan M, Based MA, Haider J. Intelligent System for Vehicles Number Plate Detection and Recognition Using Convolutional Neural Networks. Technologies. 2021; 9(1):9. https://doi.org/10.3390/technologies9010009
Chicago/Turabian StyleAlam, Nur-A-, Mominul Ahsan, Md. Abdul Based, and Julfikar Haider. 2021. "Intelligent System for Vehicles Number Plate Detection and Recognition Using Convolutional Neural Networks" Technologies 9, no. 1: 9. https://doi.org/10.3390/technologies9010009