Cognition based Recognition of Partially Occluded Traffic Signs

Document Type : Article

Authors

1 - Department of Electrical Engineering, NFC Institute of Engineering and Technology, Multan, 59060, Pakistan. - Department of Electrical Engineering, University of Engineering and Technology, Lahore, Pakistan

2 Department of Electrical Engineering, University of Engineering and Technology, Lahore, Pakistan

3 Department of Electrical Engineering, NFC Institute of Engineering and Technology, Multan, 59060, Pakistan

Abstract

Computer vision based traffic sign detection and recognition is an active field of research but the task becomes challenging when the sign of interest is partially occluded by nearby objects like a tree, pole or vehicle. Another difficulty posed especially in the developing countries is the lost colors problem that arises due to aging and poor maintenance. This work presents an automatic technique that focuses on visible parts only and suppresses occluded portions. Features are collected using a convolutional neural network inspired invariant feature extraction technique augmented with feature interaction based dimensionality reduction. Further, with the use of dynamic parameter estimation, an adaptive system for continuous learning is also proposed. Since the effect of partial occlusion has not been thoroughly studied, there is no benchmark database available for this purpose. We have prepared two datasets by combining originally and synthetically occluded images taken from field surveys and from famous GTSRB database. Experiments revealed that our technique outperforms state of the art recognition methods previously used for visible and occluded signs by obtaining 0.81 precision and 0.79 recall values on the average. The proposed method also shows a remarkably low error rate as the amount of occlusion is increased.

Keywords

References

References:
1. Gudigar, A., Chokkadi, S., and Raghavendra, U. "A review on automatic detection and recognition of traffic sign", Multimedia Tools and Applications, 75(1), pp. 333-364 (2016).
2. Noon, S., Javed, K., Mannan, A., et al. "Recognizing traffic signs using flexible discrete cosine transform (dct) grid", Scientia Iranica, Transaction D, Computer Science & Electrical Engineering, 24(3), pp. 1384-1394 (2017).
3. Bayoudh, K., Hamdaoui, F., and Mtibaa, A. "Transfer learning based hybrid 2d-3d cnn for traffic sign recognition and semantic road detection applied in advanced driver assistance systems", Applied Intelligence, 51(1), pp. 124-142 (2021).
4. Fan, B.B. and Yang, H. "Multi-scale traffic sign detection model with attention", Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 235(2-3), pp. 708-720 (2021).
5. Jin, Y., Fu, Y., Wang, W., et al. "Multi-feature fusion and enhancement single shot detector for traffic sign recognition", IEEE Access, 8, pp. 38931-38940 (2020).
6. Cao, J., Zhang, J., and Huang, W. "Traffic sign detection and recognition using multi-scale fusion and prime sample attention", IEEE Access, 9, pp. 3579- 3591 (2020).
7. Haque, W.A., Arefin, S., Shihavuddin, A., et al. "Deepthin: A novel lightweight cnn architecture for traffic sign recognition without gpu requirements", Expert Systems with Applications, 168, p. 114481 (2021).
8. Yazdan, R. and Varshosaz, M. "Improving traffic sign recognition results in urban areas by overcoming the impact of scale and rotation", ISPRS Journal of Photogrammetry and Remote Sensing, 171, pp. 18-35 (2021).
9. Cao, J., Zhang, J., and Jin, X. "A traffic-sign detection algorithm based on improved sparse r-cnn", IEEE Access, 9, pp. 122774-122788 (2021).
10. Liu, Z., Qi, M., Shen, C., et al. "Cascade saccade machine learning network with hierarchical classes for traffic sign detection", Sustainable Cities and Society, 67, p. 102700 (2021).
11. Liu, C., Chang, F., Chen, Z., et al. "Fast traffic sign recognition via high-contrast region extraction and extended sparse representation", IEEE Transactions on Intelligent Transportation Systems, 17(1), pp. 79- 92 (2015).
12. Yang, Y., Luo, H., Xu, H., et al. "Towards realtime traffic sign detection and classification", IEEE Transactions on Intelligent Transportation Systems, 17(7), pp. 2022-2031 (2016).
13. Wang, D., Hou, X., Xu, J., et al. "Traffic sign detection using a cascade method with fast feature extraction and saliency test", IEEE Transactions on Intelligent Transportation Systems, 18(12), pp. 3290-3302 (2017).
14. Greenhalgh, J. and Mirmehdi, M. "Real-time detection and recognition of road traffic signs", IEEE Transactions on Intelligent Transportation Systems, 13(4), pp. 1498-1506 (2012).
15. Liu, C., Chang, F., and Liu, C. "Occlusion-robust traffic sign detection via cascaded colour cubic feature", IET Intelligent Transport Systems, 10(5), pp. 354-360 (2016).
16. Gudigar, A., Chokkadi, S., Raghavendra, U., et al. "Local texture patterns for traffic sign recognition using higher order spectra", Pattern Recognition Letters, 94, pp. 202-210 (2017).
17. Peng, H., Long, F., and Ding, C. "Feature selection based on mutual information criteria of maxdependency, max-relevance, and min-redundancy", IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), pp. 1226-1238 (2005).
18. Guyon, I. and Elisseeff, A. "An introduction to variable and feature selection", Journal of Machine Learning Research, 3(Mar), pp. 1157-1182 (2003).
19. Scholkopf, B. and Smola, A.J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, 2nd Ed., pp. 230-259, MIT Press, USA (2002).
20. Liu, C., Li, S., Chang, F., et al. "Supplemental boosting and cascaded convnet based transfer learning structure for fast traffic sign detection in unknown application scenes", Sensors, 18(7), p. 2386 (2018).
21. Lai, Y., Wang, N., Yang, Y., et al. "Traffic signs recognition and classification based on deep feature learning", International Conference on Pattern Recognition Applications and Methods (ICPRAM), Funchal, Portugal, pp. 622-629 (2018).
22. Goodfellow, I., Bengio, Y., Courville, A., et al. Deep Learning, 1st Ed., pp. 91-150, MIT Press, USA (2016).
23. Lee, S.G., Sung, Y., Kim, Y.G., et al. "Variations of alexnet and googlenet to improve Korean character recognition performance", Journal of Information Processing Systems, 14(1), pp. 205-217 (2018).
24. Rosario, G., Sonderman, T., and Zhu, X. "Deep transfer learning for traffic sign recognition", IEEE International Conference on Information Reuse and Integration (IRI), Salt Lake City, Utah, pp. 178-185 (2018).
25. Tian, J. and Li, Y. "Convolutional neural networks for steganalysis via transfer learning", International Journal of Pattern Recognition and Artificial Intelligence, 33(2), p. 1959006 (2018).
26. Fleyeh, H. and Davami, E. "Eigen-based traffic sign recognition", IET Intelligent Transport Systems, 5(3), pp. 190-196 (2011).
27. Hou, Y.L., Hao, X., and Chen, H. "A cognitively motivated method for classification of occluded traffic signs", IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(2), pp. 255-262 (2017).
28. Mannan, A., Javed, K., and Noon, S.K. "Statistical boosting: A preprocessing technique to enhance performance of machine learning and deep learning models on partially occluded traffic signs", IEEE 23rd International Multitopic Conference (INMIC), Bahawalpur, Pakistan, pp. 1-6 (2020).
29. Mogelmose, A., Trivedi, M.M., and Moeslund, T.B. "Vision based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey", IEEE Transactions on Intelligent Transportation Systems, 13(4), pp. 1484-1497 (2012).
30. Wang, X., Han, T.X., and Yan, S. "An hog-lbp human detector with partial occlusion handling", 12th International Conference on Computer Vision, Kyoto, Japan, pp. 32-39 (2009).
31. Rehman, Y., Riaz, I., Fan, X., et al. "D-patches: effective traffic sign detection with occlusion handling", IET Computer Vision, 11(5), pp. 368-377 (2017).
32. Floros, G., Kyritsis, K., and Potamianos, G. "Database and baseline system for detecting degraded traffic signs in urban environments", 5th European Workshop on Visual Information Processing (EUVIP), Paris, France, pp. 1-5 (2014).
33. Li, L. and Ma, G. "Recognition of degraded traffic sign symbols using pnn and combined blur and affine invariants", Fourth International Conference on Natural Computation, Jinan, China, pp. 515-520 (2008).
34. Ishida, H., Takahashi, T., Ide, I., et al. "Identification of degraded traffic sign symbols by a generative learning method", 18th International Conference on Pattern Recognition (ICPR), Hong Kong, China, pp. 531-534 (2006).
35. De La Escalera, A., Armingol, J.M., Pastor, J.M., et al. "Visual sign information extraction and identification by deformable models for intelligent vehicles", IEEE Transactions on Intelligent Transportation Systems, 5(2), pp. 57-68 (2004).
36. Kim, J. "Detection of traffic signs based on eigencolor model and saliency model in driver assistance systems", International Journal of Automotive Technology, 14(3), pp. 429-439 (2013).
37. Tsai, L.W., Hsieh, J.W., Chuang, C.H., et al. "Road sign detection using eigen colour", IET Computer Vision, 2(3), pp. 164-177 (2008).
38. Ohta, Y.I., Kanade, T., and Sakai, T. "Color information for region segmentation", Computer Graphics and Image Processing, 13(3), pp. 222-241 (1980).
39. Alpaydin, E., Introduction to Machine Learning, 1st Ed., MIT Press, Istanbul, Turkey, pp. 57-103 (2004).
40. Maldonado-Bascon, S., Lafuente-Arroyo, S., Gil-Jimenez, P., et al. "Road-sign detection and recognition based on support vector machines", IEEE Transactions on Intelligent Transportation Systems, 8(2), pp. 264-278 (2007).
41. Dabbaghchian, S., Ghaemmaghami, M.P., and Aghagolzadeh, A. "Feature extraction using discrete cosine transform and discrimination power analysis with a face recognition technology", Pattern Recognition, 43(4), pp. 1431-1440 (2010).
42. Ayyalasomayajula, P., Grassi Pauletti, S., and Farine, P.A. "Retrieval of occluded images using dct phase and region merging", 19th IEEE International Conference on Image Processing, Orlando, FL, USA, pp. 2441- 2444 (2012).
43. Chen, W., Er, M.J., andWu, S. "Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain", IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 36(2), pp. 458-466 (2006).
44. Fracastoro, G., Fosson, S.M., and Magli, E. "Steerable discrete cosine transform", IEEE Transactions on Image Processing, 26(1), pp. 303-314 (2017).
45. Muller, J. "The hadamard multiplication theorem and applications in summability theory", Complex Variables and Elliptic Equations, 18(3-4), pp. 155-166 (1992).
46. Jia, H., Wang, W., Wang, D., et al. "Speech enhancement using modified mmse-lsa and phase reconstruction in voiced and unvoiced speech", International Journal of Pattern Recognition and Artificial Intelligence, 33(2), p. 1958002 (2018).
47. Gonzalez, R.C. and Woods, R.E., Digital Image Processing, 3rd Ed., Pearson Education, New Delhi, India, pp. 645-660 (2009).
48. Strang, G., Strang, G., Strang, G., et al., Introduction to Linear Algebra, 1st Ed., Cambridge Press,Wellesley, MA, pp. 60-160 (1993).
49. Jayaprakash, A. and KeziSelvaVijila, C. "Feature selection using ant colony optimization (aco) and road sign detection and recognition (rsdr) system", Cognitive Systems Research, 58, pp. 123-133 (2019).
50. Bennasar, M., Hicks, Y., and Setchi, R. "Feature selection using joint mutual information maximisation", Expert Systems with Applications, 42(22), pp. 8520- 8532 (2015).
51. Brown, G., Pocock, A., Zhao, M.J., et al. "Conditional likelihood maximisation: a unifying framework for information theoretic feature selection", Journal of Machine Learning Research, 13(Jan), pp. 27-66 (2012).
52. Zeng, Z., Zhang, H., Zhang, R., et al. "A novel feature selection method considering feature interaction", Pattern Recognition, 48(8), pp. 2656-2666 (2015).
53. Shabbir, Y., Khokhar, M.F., Shaiganfar, R., et al. "Spatial variance and assessment of nitrogen dioxide pollution in major cities of Pakistan along n5- highway", Journal of Environmental Sciences, 43, pp. 4-14 (2016).
54. Wang, C.W. and You, W.H. "Boosting-svm: effective learning with reduced data dimension", Applied Intelligence, 39(3), pp. 465-474 (2013).
55. Chang, C.C. and Lin, C.J. "Libsvm: a library for support vector machines", ACM Transactions on Intelligent Systems and Technology, 2(3), p. 27 (2011).
56. Montgomery, D.C., Runger, G.C., and Hubele, N.F., Engineering Statistics, 5th Ed., John Wiley & Sons, New Delhi, India, pp. 33-89 (2009).
57. Mardia, K. "Assessment of multinormality and the robustness of hotelling's t2 test", Applied Statistics, 24(2), pp. 163-171 (1975).
58. Yuan, X., Hao, X., Chen, H., et al. "Robust traffic sign recognition based on color global and local oriented edge magnitude patterns", IEEE Transactions on Intelligent Transportation Systems, 15(4), pp. 1466-1477 (2014).
Scientia Iranica
Volume 29, Issue 4
Transactions on Computer Science & Engineering and Electrical Engineering (D)
July and August 2022
Pages 1881-1897

Files

History

  • Receive Date: 09 January 2020
  • Revise Date: 15 December 2021
  • Accept Date: 14 February 2022

Share

How to cite

Statistics