ML-CK-ELM: An efficient multi-layer extreme learning machine using combined kernels for multi-label classification

Document Type : Article

Authors

1 Department of Computer Engineering, Shahid Bahonar University of Kerman, Kerman, Iran

2 Department of Applied Mathematics, Faculty of Sciences and Modern Technologies, Graduate University of Advanced Technology, Kerman, Iran

Abstract

Recently many neural network methods have been proposed for multi-label classification in the literature. One of these recent researches is the multi-layer extreme learning machines (ML-ELM) in which stack auto encoders have been used for tuning the weights. However, ML-ELM suffers from three primary drawbacks: First, input weights and biases are chosen randomly. Second, the pseudo-inverse solution for calculating output weights will cause to increase the reconstruction error. Third, memory and execution time of transformation matrices are proportional to the number of hidden layers. In this paper multi-layer kernel extreme learning machine, that uses a linear combination of base kernels in each layer, is proposed for multi-label classification. The proposed approach effectively addresses the above-mentioned drawbacks. Furthermore, multi-label classification data inherently have multi-modal aspects due to the variety of labels assigned to each instance. Applying a combination of different kernels is the added advantage of ML-CK-ELM that leads to implicitly assess the inherent multi-modal aspects of multi-label data; each kernel can be effectively used to cover one of the modals better than the other kernels. The empirical study indicates that ML-CK-ELM represents competitive performance against other state-of-the-art methods, and experimental results over multi-label datasets verify the feasibility of ML-CK-ELM.

Keywords

References

  1. References:

    1. Charte, F., del Jesus, M.J., and Rivera, A.J. Multilabel classification: Problem analysis, metrics and techniques", In Multilabel Classification: Problem Analysis, Metrics and Techniques, Springer (2016).
    2. Mercan, C., Aksoy, S., Mercan, E., Shapiro, L.G., Weaver, D.L., and Elmore, J.G. Multi-instance multilabel learning for multi-class classi_cation of whole slide breast histopathology images", IEEE Transactions on Medical Imaging, 37(1), pp. 316{325 (2017).
    3. Gibaja, E. and Ventura, S. Multi-label learning: a review of the state of the art and ongoing research", Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 4(6), pp. 411{444 (2014).
    4. Zhang, M.-L. and Zhou, Z.-H. A review on multilabel learning algorithms", IEEE Transactions on Knowledge and Data Engineering, 26(8), pp. 1819{ 1837 (2014).
    5. Wu, Y.-P. and Lin, H.-T. Progressive random klabelsets for cost-sensitive multi-label classi_cation", Machine Learning, 106(5), pp. 671{694 (2017).
    6. Trajdos, P. and Kurzynski, M. Dynamic classifier chains for multi-label learning", In German Conference on Pattern Recognition, pp. 567{580 (2019). 7. Zhang, M.-L. and Zhou, Z.-H. ML-KNN: A lazy learning approach to multi-label learning", Pattern Recognition, 40(7), pp. 2038{2048 (2007). 8. Zhang, M.-L. M l-rbf: Rbf neural networks for multilabel learning", Neural Processing Letters, 29(2), pp. 61{74 (2009). 9. Deng, W.-Y., Zheng, Q.-H., Chen, L., and Xu, X.- B. Research on extreme learning of neural networks", Chinese Journal of Computers, 33(2), pp. 279{287 (2010). 10. Zhang, N., Ding, S., and Zhang, J. Multi layer ELMRBF for multi-label learning", Applied Soft Computing, 43, pp. 535{545 (2016). 11. Kasun, L.L.C., Yang, Y., Huang, G.-B., and Zhang, Z. Dimension reduction with extreme learning machine", IEEE Transactions on Image Processing, 25(8), pp. 3906{3918 (2016). 12. Kasun, L.L.C., Zhou, H., Huang, G.-B., and Vong, C.M. Representational learning with extreme learning machine for big data", IEEE Intelligent Systems, 28(6), pp. 31{34 (2013). 13. Tissera, M.D. and McDonnell, M.D. Deep extreme learning machines for classi_cation", In Proceedings of ELM-2014, 1, pp. 345{354, Springer (2016). 14. Tang, J., Deng, C., and Huang, G.-B. Extreme learning machine for multilayer perceptron", IEEE Transactions on Neural Networks and Learning Systems, 27(4), pp. 809{821 (2016). 15. Huang, G.-B., Zhou, H., Ding, X., and Zhang, R. Extreme learning machine for regression and multiclass classi_cation", IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(2), pp. 513{ 529 (2012). 16. Wong, C.M., Vong, C.M., Wong, P.K., and Cao, J. Kernel-based multilayer extreme learning machines for representation learning", IEEE Transactions on Neural Networks and Learning Systems, 29(3), pp. 757{762 (2018). 17. Chen, T., Wang, S., and Chen, S. Deep multimodal network for multi-label classi_cation", In 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 955{960 (2017). 18. Huang, Y., Wang, W., and Wang, L. Unconstrained multimodal multi-label learning", IEEE Transactions on Multimedia, 17(11), pp. 1923{1935 (2015). 3018 M. Rezaei Ravari et al./Scientia Iranica, Transactions D: Computer Science & ... 27 (2020) 3005{3018 19. Li, K., Zou, C., Bu, S., Liang, Y., Zhang, J., and Gong, M. Multi-modal feature fusion for geographic image annotation", Pattern Recognition, 73, pp. 1{14 (2018). 20. Song, L., Liu, J., Qian, B., Sun, M., Yang, K., Sun, M., et al. A deep multi-modal CNN for multi-instance multi-label image classi_cation", IEEE Transactions on Image Processing, 27(12), pp. 6025{6038 (2018). 21. Garc__a, S., Fern_andez, A., Luengo, J., and Herrera, F. Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power", Information Sciences, 180(10), pp. 2044{2064 (2010).
Scientia Iranica
Volume 27, Issue 6 - Serial Number 6
Transactions on Computer Science & Engineering and Electrical Engineering (D)
November and December 2020
Pages 3005-3018

Files

Share

How to cite

Statistics