ML-CK-ELM: An efficient multi-layer extreme learning machine using combined kernels for multi-label classification

Document Type : Article


1 Department of Computer Engineering, Shahid Bahonar University of Kerman, Kerman, Iran

2 Department of Applied Mathematics, Faculty of Sciences and Modern Technologies, Graduate University of Advanced Technology, Kerman, Iran


Recently many neural network methods have been proposed for multi-label classification in the literature. One of these recent researches is the multi-layer extreme learning machines (ML-ELM) in which stack auto encoders have been used for tuning the weights. However, ML-ELM suffers from three primary drawbacks: First, input weights and biases are chosen randomly. Second, the pseudo-inverse solution for calculating output weights will cause to increase the reconstruction error. Third, memory and execution time of transformation matrices are proportional to the number of hidden layers. In this paper multi-layer kernel extreme learning machine, that uses a linear combination of base kernels in each layer, is proposed for multi-label classification. The proposed approach effectively addresses the above-mentioned drawbacks. Furthermore, multi-label classification data inherently have multi-modal aspects due to the variety of labels assigned to each instance. Applying a combination of different kernels is the added advantage of ML-CK-ELM that leads to implicitly assess the inherent multi-modal aspects of multi-label data; each kernel can be effectively used to cover one of the modals better than the other kernels. The empirical study indicates that ML-CK-ELM represents competitive performance against other state-of-the-art methods, and experimental results over multi-label datasets verify the feasibility of ML-CK-ELM.


1. Charte, F., del Jesus, M.J., and Rivera, A.J. Multilabel
classi cation: Problem analysis, metrics and
techniques", In Multilabel Classi cation: Problem
Analysis, Metrics and Techniques, Springer (2016).
2. Mercan, C., Aksoy, S., Mercan, E., Shapiro, L.G.,
Weaver, D.L., and Elmore, J.G. Multi-instance multilabel
learning for multi-class classi cation of whole
slide breast histopathology images", IEEE Transactions
on Medical Imaging, 37(1), pp. 316{325 (2017).
3. Gibaja, E. and Ventura, S. Multi-label learning: a
review of the state of the art and ongoing research",
Wiley Interdisciplinary Reviews: Data Mining and
Knowledge Discovery, 4(6), pp. 411{444 (2014).
4. Zhang, M.-L. and Zhou, Z.-H. A review on multilabel
learning algorithms", IEEE Transactions on
Knowledge and Data Engineering, 26(8), pp. 1819{
1837 (2014).
5. Wu, Y.-P. and Lin, H.-T. Progressive random klabelsets
for cost-sensitive multi-label classi cation",
Machine Learning, 106(5), pp. 671{694 (2017).
6. Trajdos, P. and Kurzynski, M. Dynamic classi er
chains for multi-label learning", In German Conference
on Pattern Recognition, pp. 567{580 (2019).
7. Zhang, M.-L. and Zhou, Z.-H. ML-KNN: A lazy
learning approach to multi-label learning", Pattern
Recognition, 40(7), pp. 2038{2048 (2007).
8. Zhang, M.-L. M l-rbf: Rbf neural networks for multilabel
learning", Neural Processing Letters, 29(2), pp.
61{74 (2009).
9. Deng, W.-Y., Zheng, Q.-H., Chen, L., and Xu, X.-
B. Research on extreme learning of neural networks",
Chinese Journal of Computers, 33(2), pp. 279{287
10. Zhang, N., Ding, S., and Zhang, J. Multi layer ELMRBF
for multi-label learning", Applied Soft Computing,
43, pp. 535{545 (2016).
11. Kasun, L.L.C., Yang, Y., Huang, G.-B., and Zhang,
Z. Dimension reduction with extreme learning machine",
IEEE Transactions on Image Processing,
25(8), pp. 3906{3918 (2016).
12. Kasun, L.L.C., Zhou, H., Huang, G.-B., and Vong,
C.M. Representational learning with extreme learning
machine for big data", IEEE Intelligent Systems,
28(6), pp. 31{34 (2013).
13. Tissera, M.D. and McDonnell, M.D. Deep extreme
learning machines for classi cation", In Proceedings of
ELM-2014, 1, pp. 345{354, Springer (2016).
14. Tang, J., Deng, C., and Huang, G.-B. Extreme
learning machine for multilayer perceptron", IEEE
Transactions on Neural Networks and Learning Systems,
27(4), pp. 809{821 (2016).
15. Huang, G.-B., Zhou, H., Ding, X., and Zhang, R. Extreme
learning machine for regression and multiclass
classi cation", IEEE Transactions on Systems, Man,
and Cybernetics, Part B (Cybernetics), 42(2), pp. 513{
529 (2012).
16. Wong, C.M., Vong, C.M., Wong, P.K., and Cao, J.
Kernel-based multilayer extreme learning machines
for representation learning", IEEE Transactions on
Neural Networks and Learning Systems, 29(3), pp.
757{762 (2018).
17. Chen, T., Wang, S., and Chen, S. Deep multimodal
network for multi-label classi cation", In 2017 IEEE
International Conference on Multimedia and Expo
(ICME), pp. 955{960 (2017).
18. Huang, Y., Wang, W., and Wang, L. Unconstrained
multimodal multi-label learning", IEEE Transactions
on Multimedia, 17(11), pp. 1923{1935 (2015).
3018 M. Rezaei Ravari et al./Scientia Iranica, Transactions D: Computer Science & ... 27 (2020) 3005{3018
19. Li, K., Zou, C., Bu, S., Liang, Y., Zhang, J., and Gong,
M. Multi-modal feature fusion for geographic image
annotation", Pattern Recognition, 73, pp. 1{14 (2018).
20. Song, L., Liu, J., Qian, B., Sun, M., Yang, K., Sun,
M., et al. A deep multi-modal CNN for multi-instance
multi-label image classi cation", IEEE Transactions
on Image Processing, 27(12), pp. 6025{6038 (2018).
21. Garca, S., Fernandez, A., Luengo, J., and Herrera, F.
Advanced nonparametric tests for multiple comparisons
in the design of experiments in computational
intelligence and data mining: Experimental analysis of
power", Information Sciences, 180(10), pp. 2044{2064