Iranian license plate recognition using a reliable deep learning approach

Document Type : Article

Authors

Faculty of Mechanical Engineering, Tarbiat Modares University, Tehran, Iran

Abstract

The issue of Automatic License Plate Recognition (ALPR) has been a challenging one in recent years, with various factors such as weather conditions, camera angle, lighting, and different characters on license plates. However, thanks to the advances made in deep neural networks, it is now possible to use specific types of neural networks and models to recognize Iranian license plates. In the proposed method, license plate recognition is done in two steps. Firstly, the license plates are detected from the input image using the YOLOv4-tiny model, which is based on the Convolutional Neural Network (CNN). Secondly, the characters on the license plates are recognized using the Convolutional Recurrent Neural Network (CRNN) and Connectionist Temporal Classification (CTC). With no need to segment and label the characters separately, one string of numbers and letters is enough for the labels. The successful training of the models involved using 3065 images of license plates and 3364 images of license plate characters as the desired datasets. The proposed method boasts an average response time of 0.0074 seconds per image and 141 frames per second (fps) in the Darknet framework and 0.128 seconds per image in the TensorFlow framework for the License Plate Detection (LPD) part.

Keywords

Main Subjects

References

References:
1. Luo, S. and Liu, J. "Research on car license plate recognition based on improved YOLOv5m and LPRNet", IEEE Access, 10, pp. 93692-93700 (2022). DOI: 10.1109/ACCESS.2022.3203388.
2. Laroca, R., Santos, M., Estevam, V., et al. "A first look at dataset bias in license plate recognition", In 35th Conference on Graphics, Patterns, and Images, SIBGRAPI 2022, (Aug. 2022). DOI: 10.1109/sibgrapi55357.2022.9991768.
3. Deng, J., Dong, W., Socher, R., et al. "Imagenet: A large-scale hierarchical image database", In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255 (2009). DOI: 10.1109/CVPR.2009.5206848.
4. Lin, T.-Y., Maire, M., Belongie, S., et al. "Microsoft coco: Common objects in context", In European Conference on Computer Vision, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds., Cham: Springer International Publishing, pp. 740-755 (2014). DOI: 10.1007/978-3-319-10602-1-48.
5. Everingham, M., Van Gool, L., Williams, C.K.I., et al. "The pascal visual object classes (voc) challenge", Int J Comput Vis, 88(2), pp. 303-338 (2010). DOI: 10.1007/s11263-009-0275-4.
6. Khoramdel, J., Hatami, S., and Sadedel, M. "Wearing face mask detection using deep learning during COVID-19 pandemic", Scientia Iranica, 30(3), pp. 1058-1067 (2023). DOI: 10.24200/sci.2023.59093.6057.
7. Xu, Z., Yang, W., Meng, A., et al. "Towards endto- end license plate detection and recognition: A large dataset and baseline," In Proceedings of the European Conference on Computer Vision (ECCV), Journal of Electronic Imaging, 25, pp. 255-271 (2018). https://doi.org/10.1007/978-3-030-01261-8-16.
8. Goncalves, G., Silva, S., Menotti, D., et al. "Benchmark for license plate character segmentation", 25(5), (Jul. 2016). DOI: 10.1117/1.jei.25.5.053034.
9. Tourani, A., Shahbahrami, A., Soroori, S., et al. "A robust deep learning approach for automatic iranian vehicle license plate detection and recognition for surveillance systems", IEEE Access, 8, pp. 201317- 201330 (2020). DOI: 10.1109/ACCESS.2020.3035992.
10. Rakhshani, S., Rashedi, E., and Nezamabadi-pour, H. "License plate recognition using deep learning", Journal of Machine Vision and Image Processing, 6(1), pp. 31-46 (2019). DOI: 20.1001.1.23831197.1398.6.1.3.3.
11. Tourani, A., Soroori, S., Shahbahrami, A., et al. "Iranis: A large-scale dataset of farsi license plate characters", In 5th International Confrence on Pattern Recognition and Image Analysis, pp. 1-5 (2021). DOI: 10.1109/IPRIA53572.2021.9483461.
12. Rahmani, M., Sabaghian, M., Moghadami, S.M., et al. "IR-LPR: Large scale of iranian license plate recog1120 S. Hatami et al./Scientia Iranica, Transactions D: Computer Science & ... 31 (2024) 1105-1121 nition dataset", In 12th International Conference on Computer and Knowledge Engineering, ICCKE 2022, (Sep. 2022). DOI: 10.48550/arxiv.2209.04680.
13. Alzubaidi L., Zhang, J., Humaidi, A.J., et al. "Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions", J Big Data, 8(1), p. 53 (2021). DOI: 10.1186/s40537-021-00444-8.
14. Girshick, R., Donahue, J., Darrell, T., et al. "Regionbased convolutional networks for accurate object detection and segmentation", IEEE Trans PatternAnal Mach Intell, 38(1), pp. 142-158 (2016). DOI: 10.1109/TPAMI.2015.2437384.
15. Girshick, R. "Fast R-CNN", In 2015 IEEE International Conference on Computer Vision, pp. 1550-5499 (Apr. 2015). 
16. Ren, S., He, K., Girshick, R., et al. "Faster RCNN: Towards real-time object detection with region proposal networks", IEEE Trans Pattern Anal Mach Intell, 39(6), pp. 1137-1149 (2015). DOI: 10.1109/TPAMI.2016.2577031.
17. Redmon, J., Divvala, S., Girshick, R., et al. "You only look once: Unified, real-time object detection", In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788 (2016). DOI: 10.1109/CVPR.2016.91.
18. Redmon, J. and Farhadi, A. "YOLO9000: Better, faster, stronger", In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016). DOI: 10.1109/CVPR.2017.690.
19. Redmon, J. and Farhadi, A. "YOLOv3: An incremental improvement", arXiv Preprint arXiv:1804.02767 (2018).
20. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. "YOLOv4: Optimal speed and accuracy of object detection", arXiv Preprint arXiv:2004.10934 (2020). 
21. Liu, W., Anguelov, D., Erhan, D., et al. "SSD: Single shot multibox detector", In European Conference on Computer Vision, pp. 21-37 (2016). DOI: 10.1007/978- 3-319-46448-0-2.
22. Shi, B., Bai, X., and Yao, C. "An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition", IEEE Trans Pattern Anal Mach Intell, (Jul, 2015). DOI: 10.1109/TPAMI.2016.2646371.
23. Graves, A., Fernandez, S., Gomez, F., et al. "Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks", In ICML 2006-Proceedings of the 23rd International Conference on Machine Learning, pp. 369-376 (Jan. 2006). DOI: 10.1145/1143844.1143891.
24. Massoud, M.A., Sabee, M., Gergais, M., et al. "Automated new license plate recognition in egypt", Alexandria Engineering Journal, 52, pp. 319-326 (Sep. 2013). DOI: 10.1016/j.aej.2013.02.005.
25. Tuba, M., Akashe, S., and Joshi, A., ICT Systems and Sustainability Proceedings of ICT4SD 2019, 1 (2019). DOI: 10.1007/978-981-15-0936-0.
26. Singh, J. and Bhushan, B. "Real time indian license plate detection using deep neural networks and optical character recognition using LSTM tesseract", In 2019 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), pp. 347-352 (2019). DOI: 10.1109/ICCCIS48478.2019.8974469.
27. Jamtsho, Y., Riyamongkol, P., and Waranusast, R. "Real-time Bhutanese license plate localization using YOLO", ICT Express (Nov. 2019). DOI: 10.1016/j.icte.2019.11.001.
28. Lin, C.-H. and Wu, C.-H. "A lightweight, highperformance multi-angle license plate recognition model", In 2019 International Conference on Advanced Mechatronic Systems (ICAMechS), pp. 235-240 (2019). DOI: 10.1109/ICAMechS.2019.8861688.
29. Montazzolli, S. and Jung, C.R. "Real-time brazilian license plate detection and recognition using deep convolutional neural networks", In 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 55-62 (2017). DOI: 10.1109/SIBGRAPI.2017.14.
30. Samadzadeh, A., Shayan, A.M., Rouhani, B., et al. "RILP: Robust iranian license plate recognition designed for complex conditions", In 2020 International Conference on Machine Vision and Image Processing (MVIP), pp. 1-7 (2020). DOI: 10.1109/MVIP49855.2020.9116910.
31. Wang, W., Yang, J., Chen, M., et al. "A light CNN for end-to-end car license plates detection and recognition", IEEE Access, 7, pp. 173875-173883 (2019). DOI: 10.1109/ACCESS.2019.2956357.
32. Montazzolli, S. and Jung, C.R. "License plate detection and recognition in unconstrained scenarios", In European Conference on Computer Vision, Springer, pp. 593-609 (2018). DOI: 10.1007/978-3-030-01258-8- 36.
33. Al Batat, R., Angelopoulou, A., Premkumar, S., et al. "An end-to-end automated license plate recognition system using YOLO based vehicle and license plate detection with vehicle classification", Sensors, 22, p. 9477 (Dec. 2022). DOI: 10.3390/s22239477.
34. Rashtehroudi, A., Shahbahrami, A., and Akoushideh, A. "Iranian license plate recognition using deep learning", In 2020 International Conference on Machine Vision and Image Processing (MVIP), pp. 1-5 (2020). DOI: 10.1109/MVIP49855.2020.9116897.
Scientia Iranica
Volume 31, Issue 14
Transactions on Computer Science & Engineering and Electrical Engineering (D)
July and August 2024
Pages 1105-1121

Files

History

  • Receive Date: 25 October 2022
  • Revise Date: 25 January 2024
  • Accept Date: 18 May 2024

Share

How to cite

Statistics