A novel approach for vehicle identification based on image registration and deep learning

Document Type : Article

Authors

1 Faculty of Electrical Engineering, Shahrood University of Technology, Shahrood, P.O. Box 3619995161, Iran

2 School of Electrical Engineering, Iran University of Science & Technology, Tehran, P.O. Box 16765-163, Iran

3 Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

Abstract

Fine-grained vehicle type recognition using on-road cameras is among interesting topics in machine vision. It has several challenges like inter-class similarity, different viewing angles, and different lighting and weather conditions. This paper presents a novel approach for vehicle classification based on a novel augmentation method and deep learning. In the proposed smart augmentation, the vehicle images of each class are registered on the reference vehicles of all other classes and then added to the training set of that class. In this way, we will have a lot of new images which are very similar to both reference and target classes. This helps the CNN model to handle inter-class similarities very well. In the test phase, the input image is registered on every reference image in parallel and applied to the model. Finally, the winner is determined by summing up the provided scores of all models. The targeted data augmentation along with the proposed classification strategy has high recognition power and is capable of providing high accuracy using small CNNs or any other classification method without the need for large datasets. The proposed method achieved a recognition rate of 99.8% with only 150K parameters.

Keywords

References

References:
1. Biglari, M., Soleimani, A., and Hassanpour, H. "A cascaded part-based system for fine-grained vehicle classification", IEEE Transactions on Intelligent Transportation Systems, 19(1), pp. 273-283 (2018). DOI: 10.1109/TITS.2017.2749961.
2. Ma, Z., Chang, D., Xie, J., et al. "Fine-grained vehicle classification with channel max pooling modified CNNs", IEEE Transactions on Vehicular Technology, 68(4), pp. 3224-3233 (2019).DOI: 10.1109/TVT.2019.2899972.
3. Wang, J., Zheng, H., Huang, Y., et al. "Vehicle type recognition in surveillance images from labeled web-nature data using deep transfer learning", IEEE Transactions on Intelligent Transportation Systems, 19(9), pp. 2913-2922 (2018). DOI: 10.1109/TITS.2017.2765676.
4. Huang, Y., Wu, R., Sun, Y., et al. "Vehicle logo recognition system based on convolutional neural networks with a pretraining strategy", IEEE Transactions on Intelligent Transportation Systems, 16(4), pp. 1951- 1960 (2015). DOI: 10.1109/TITS.2014.2387069.
5. Psyllos, A.P., Anagnostopoulos, C.-N.E., and Kayafas, E. "Vehicle logo recognition using a SIFT-based enhanced matching scheme", IEEE Transactions on Intelligent Transportation Systems, 11(2), pp. 322-328 (2010). DOI: 10.1109/TITS.2010.2042714.
6. Zhang, J., Xiao, W., Coifman, B., et al. "Vehicle tracking and speed estimation from roadside lidar", IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, pp. 5597-5608 (2020). DOI: 10.1109/JSTARS.2020.3024921.
7. Nazemi, A., Azimifar, Z., Shafiee, M.J., et al. "Realtime vehicle make and model recognition using unsupervised feature learning", IEEE Transactions on Intelligent Transportation Systems, 21(7), pp. 3080- 3090 (2020). DOI: 10.1109/TITS.2019.2924830.
8. Cai, D., Chen, K., Qian, Y., et al. "Convolutional low-resolution fine-grained classification", Pattern Recognition Letters, 119, pp. 166-171 (2019). DOI: 10.1016/j.patrec.2017.10.020.
9. Ni, X. and Huttunen, H. "Vehicle attribute recognition by appearance: Computer vision methods for vehicle type, make and model classification", Journal of Signal Processing Systems, 93(4), pp. 357-368 (2020). DOI: 10.1007/s11265-020-01567-6.
10. Soon, F.C., Khaw, H.Y., Chuah, J.H., et al. "PCANetbased convolutional neural network architecture for a vehicle model recognition system", IEEE Transactions on Intelligent Transportation Systems, 20(2), pp. 749- 759 (2019). DOI: 10.1109/TITS.2018.2833620.
11. Ren, S., He, K., Girshick, R., et al. "Faster RCNN: Towards real-time object detection with region proposal networks", IEEE Trans Pattern Anal Mach Intell, 39(6), pp. 1137-1149 (2017). DOI: 10.1109/TPAMI.2016.2577031.
12. Yang, L., Luo, P., Change Loy, C., et al. "A largescale car dataset for fine-grained categorization and verification", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3973- 3981 (2015). DOI: 10.1109/CVPR.2015.7299023.
13. Gholamalinejad, H. and Khosravi, H. "IRVD: A largescale dataset for classification of Iranian vehicles in urban streets", Journal of AI and Data Mining, 9(1), pp. 1-9 (2021). DOI: 10.22044/jadm.2020.8438.1982.
14. Yu, S., Wu, Y., Li, W., et al. "A model for finegrained vehicle classification based on deep learning", Neurocomputing, 257, pp. 97-103 (2017). DOI: 10.1016/j.neucom.2016.09.116.
15. Sochor, J., Herout, A., and Havel, J. "BoxCars: 3D boxes as CNN input for improved fine-grained vehicle recognition", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3006- 3015 (2016). DOI: 10.1109/CVPR.2016.328.
16. Sochor, J., Spanhel, J., and Herout, A. "BoxCars: Improving fine-grained recognition of vehicles using 3- D bounding boxes in traffic surveillance", IEEE Transactions on Intelligent Transportation Systems, 20(1), pp. 97-108 (2019). DOI: 10.1109/TITS.2018.2799228.
17. Huang, Y., Liang, B., Xie, W., et al. "Dual domain multi-task model for vehicle re-identification", IEEE Transactions on Intelligent Transportation Systems, 23(4), pp. 1-9 (2020). DOI: 10.1109/TITS.2020.3027578.
18. Simonyan, K. and Zisserman, A. "Very deep convolutional networks for large-scale image recognition", arXiv preprint arXiv:1409.1556. (2014). 
19. Liu, R., Yuan, Z., and Liu, T. "Learning TBox with a cascaded anchor-free network for vehicle detection", IEEE Transactions on Intelligent Transportation Systems, 23(1), pp. 1-12 (2020). DOI: 10.1109/TITS.2020.3010523.
20. Meng, D., Li, L., Liu, X., et al. "Parsing-based viewaware embedding network for vehicle re-identification", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7103-7112 (2020). DOI: 10.1109/CVPR42600.2020.00713.
21. Liu, H., Tian, Y., Yang, Y., et al. "Deep relative distance learning: Tell the difference between similar vehicles", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2167- 2175 (2016). DOI: 10.1109/CVPR.2016.238.
22. Zhu, X., Luo, Z., Fu, P., et al. "VOC-ReID: Vehicle re-identification based on vehicle-orientation-camera", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 602-603 (2020). DOI: 10.1109/CVPRW50498.2020.00309.
23. Hsieh, J.-W., Chen, L.-C., and Chen, D.-Y. "Symmetrical SURF and its applications to vehicle detection and vehicle make and model recognition", IEEE Transactions on Intelligent Transportation Systems, 15(1), pp. 6-20 (2014). DOI: 10.1109/TITS.2013.2294646.
24. Biglari, M., Soleimani, A., and Hassanpour, H. "Partbased recognition of vehicle make and model", IET Image Processing, 11(7), pp. 483-491 (2017). DOI: 10.1049/iet-ipr.2016.0969.
25. Madden, M. and Munroe, D.T., Multi-Class and Single-Class Classification Approaches to Vehicle Model Recognition from Images, Proc. AICS (2005). 
26. Boonsim, N. and Prakoonwit, S. "Car make and model recognition under limited lighting conditions at night", Pattern Analysis and Applications, 20(4), pp. 1195- 1207 (2016). DOI: 10.1007/s10044-016-0559-6.
27. Sarfraz, M.S. and Khan, M.H., A Probabilistic Framework for Patch based Vehicle Type Recognition, Visapp. 1 (2011).
28. Asgarian Dehkordi, R. and Khosravi, H. "Vehicle type recognition based on dimension estimation and bag of word classification", Journal of AI and Data Mining, 8(3), pp. 427-438 (2020). DOI: 10.22044/JADM.2020.8375.1975.
29. Howard, A.G., Zhu, M., Chen, B., et al. "Mobilenets: Efficient convolutional neural networks for mobile vision applications", arXiv preprint arXiv:1704.04861. (2017). DOI: 10.48550/arXiv.1704.04861.
30. Tan, M. and Le, Q. "Efficientnet: Rethinking model scaling for convolutional neural networks", International Conference on Machine Learning. PMLR, pp. 6105-6114 (2019).
31. Nagy, B., Foehn, P., and Scaramuzza, D., Faster than FAST: GPU-Accelerated Frontend for High-Speed VIO, arXiv preprint arXiv:2003.13493. (2020). DOI: 10.1109/IROS45743.2020.9340851.
Scientia Iranica
Volume 31, Issue 5 - Serial Number 5
Transactions on Computer Science & Engineering and Electrical Engineering (D)
March and April 2024
Pages 431-440

Files

Share

How to cite

Statistics