Domain Adaptation via Bregman divergence minimization

Document Type : Article

Authors

Faculty of Information Technology and Computer Engineering, Urmia University of Technology, Urmia, Iran

Abstract

In recent years, Fisher linear discriminant analysis (FLDA) based classifi-
cation is one of the most successful approaches and has been shown effective
performance in different classification tasks. However, when the learning data
(source domain) have a different distribution against the testing data (tar-
get domain), the FLDA-based models may not be optimal, and the perfor-
mance will be degraded, dramatically. To face this problem, in this paper, we
propose an optimal domain adaptation via Bregman divergence minimization
(DAB) approach, in which the discriminative features of source and target do-
mains are simultaneously learned via domain invariant representation. DAB
is designed based on the constraints of FLDA, with the objective to adapt
the coupled marginal and conditional distribution mismatches with Breg-
man divergence minimization. Thus, the resulting representation can have
well functionality like FLDA and simultaneously have better discrimination
ability. Moreover, our proposed approach can be easily kernelized to deal
with nonlinear tasks. Extensive experiments on various benchmark datasets
show that our DAB can effectively deal with the cross domain divergence and
outperforms several state-of-the-art domain adaptation approaches on cross-
distribution domains.

Keywords

References

References:
1. Zandifar, M. and Tahmoresnezhad, J. "Locality Fisher discriminant analysis for conditional domain adaption", Iran Journal of Computer Science, 4(1), pp. 17-34 (2020).
2. Si, S., Tao, D., and Geng, B. "Bregman divergencebased regularization for transfer subspace learning", IEEE T KNOWL DATA EN, 22(7), pp. 929-942 (2010).
3. Karimpour, M., Saray, S.N., Tahmoresnezhad, J., et al. "Multi-source domain adaptation for image classification", Machine Vision and Applications, 31(6), pp. 1-19 (2020).
4. Tahmoresnezhad, J. and Hashemi, S. "An efficient yet effective random partitioning and feature weighting approach for transfer learning", International Journal of Pattern Recognition and Artificial Intelligence, 30(02), p. 1651003 (2016).
5. Vapnik, V.N., Lect Notes Math., Wiley (1998).
6. Cayton, L. "Fast nearest neighbor retrieval for bregman divergences", In Proceedings of the 25th International Conference on Machine Learning, pp. 112-119 (2008).
7. Noble, W.S. "What is a support vector machine", Nature Biotechnology, 24(12), pp. 1565-1567, (2006).
8. Denisov, P. and Vu, N.T. "End-to-end multi-speaker speech recognition using speaker embeddings and transfer learning", arXiv Preprint arXiv:1908.04737 (2019).
9. Shivakumar, P.G. and Georgiou, P. "Transfer learning from adult to children for speech recognition: Evaluation, analysis and recommendations", Computer Speech and Language, 63, p. 101077 (2020).
10. Liu, R., Shi, Y., Ji, C. et al. "A survey of sentiment analysis based on transfer learning", IEEE Access, 7, pp. 85401-85412 (2019).
11. Wei, W., Meng, D., Zhao, Q., Xu, Z., et al. "Semisupervised transfer learning for image rain removal", In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3877-3886 (2019).
12. Saray, S.N. and Tahmoresnezhad, J. "Joint distinct subspace learning and unsupervised transfer classification for visual domain adaptation", Signal, Image and Video Processing, pp. 1-9 (2020).
13. Wang, J., Zheng, V. W., Chen, Y., et al. "Deep transfer learning for cross-domain activity recognition", In Proceedings of the 3rd International Conference on Crowd Science and Engineering, pp. 1-8 (2018).
14. Zhu, H., Samtani, S., Chen, H., et al. "Human identification for activities of daily living: A deep transfer learning approach", Journal of Management Information Systems, 37(2), pp. 457-483 (2020).
15. Cai, L., Gu, J., Ma, J., et al. "Probabilistic wind power forecasting approach via instance-based transfer learning embedded gradient boosting decision trees", Energies, 12(1), p. 159, (2019).
16. Sun, G. , Liang, L., Chen, T., et al. "Network traffic classification based on transfer learning", Computers and Electrical Engineering, 69, pp. 920-927 (2018).
17. Hooshmand, A. and Sharma, R. "Energy predictive models with limited data using transfer learning", In Proceedings of the Tenth ACM International Conference on Future Energy Systems, pp. 12-16 (2019).
18. Zhong, X., Guo, S., Shan, H., et al. "Feature-based transfer learning based on distribution similarity", IEEE Access, 6, pp. 35551-35557 (2018).
19. Gholenji, E. and Tahmoresnezhad, J. "Joint local and statistical discriminant learning via feature alignment", Signal, Image and Video Processing, 14(3), pp. 1-8 (2019).
20. Gong, B., Grauman, K., and Sha, F. "Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation", ASTR SOC P, pp. 222-230 (2013).
21. Xiao, T., Liu, P., Zhao W., et al. "Iterative landmark selection and subspace alignment for unsupervised domain adaptation", Journal of Electronic Imaging, 27(3), p. 033037 (2018).
22. Aljundi, R., Emonet, R., Muselet, D., et al. "Landmarks-based kernelized subspace alignment for unsupervised domain adaptation", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 56-63 (2015).
23. Wang, B., Qiu, M., Wang, X., et al. "A minimax game for instance based selective transfer learning", in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining ACM, pp. 34-43 (2019).
24. Long, M., Wang, J., Ding, G., et al. "Adaptation regularization: A general framework for transfer learning", IEEE Transactions on Knowledge and Data Engineering, 26(5), p. 1076-1089 (2014).
25. Gheisari, M. and Baghshah, M.S. "Unsupervised domain adaptation via representation learning and adaptive classifier learning", Neurocomputing, 165, pp. 300-311 (2015).
26. Gheisari, M. and Baghshah, M.S. "Joint predictive model and representation learning for visual domain adaptation", Eng Appl Artif Intel, 58, pp. 157-170 (2017).
27. Fodor, I.K. "A survey of dimension reduction techniques", Cmr Worksh, 9, pp. 1-18 (2002).
28. Tao, D., Li, X., Wu, X., et al. "Geometric mean for subspace selection", IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2), pp. 260-274 (2008).
29. Song, P., Zheng, W., Liu, J., et al. "A novel speech emotion recognition method via transfer PCA and sparse coding", In Chinese Conference on Biometric Recognition, pp. 393-400 (2015).
30. Song, S., Yu, H., Miao, Z., et al. "Domain adaptation for convolutional neural networks-based remote sensing scene classification", IEEE Geoscience and Remote Sensing Letters, 16(8), pp. 1324-1328 (2019).
31. Tahmoresnezhad, J. and Hashemi, S. "Visual domain adaptation via transfer feature learning", Knowl Inf Syst, 50(2), pp. 585-605 (2017).
32. Liu, J., Li, J., and Lu, K. "Coupled local-global adaptation for multi-source transfer learning", Neurocomputing, 275, pp. 247-254 (2018).
33. Rezaei, S. and Tahmoresnezhad, J. "Discriminative and domain invariant subspace alignment for visual tasks", Iran Journal of Computer Science, 2(4) pp. 219-230 (2019).
34. Ghifary, M., Balduzzi, D., Kleijn, W.B., et al. "Scatter component analysis: A unified framework for domain adaptation and domain generalization", IEEE T Pattern Anal, 39(7), pp. 1414-1430, (2017).
35. Gneiting, T., Balabdaoui, F. and Raftery, A.E.  Probabilistic forecasts, calibration and sharpness", J R Stat Soc, 69(2), pp. 243-268 (2007).
36. Wand M.P. and Jones, M.C., Kernel Smoothing, Crc Press (1994).
37. Pan, S.J., Kwok, J.T., and Yang, Q. "Transfer learning via dimensionality reduction", AAAS R&D B, 8, pp. 677-682 (2008).
38. Wang, J., Chen, Y., Hao, S., et al. "Balanced distribution adaptation for transfer learning", In 2017 IEEE International Conference on Data Mining (ICDM), pp. 1129-1134 (2017).
39. Torkkola, K. "Feature extraction by non-parametric mutual information maximization", J Mach Learn Res, 3(Mar), pp. 1415-1438 (2003).
40. Gong, B., Shi, Y., Sha, F., et al. "Geodesic  flow kernel for unsupervised domain adaptation", 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2066-2073 (2012).
41. Long, M., Wang, J., Ding, G., et al. "Transfer feature learning with joint distribution adaptation", Lect Notes Comput Sc, pp. 2200-2207 (2013).
42. Long, M., Wang, J., Ding, G., et al. "Transfer joint matching for unsupervised domain adaptation", 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1410-1417 (2014).
43. Xu, Y., Fang, X., Wu, J., et al. "Discriminative transfer subspace learning via low-rank and sparse representation", IEEE T Image Process, 25(2), pp. 850-863, (2016).
44. Zhang, W. and Wu, D. "Discriminative joint probability maximum mean discrepancy (DJP-MMD) for domain adaptation", In 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, IEEE (2020).
Scientia Iranica
Volume 28, Issue 6 - Serial Number 6
Transactions on Computer Science & Engineering and Electrical Engineering (D)
November and December 2021
Pages 3273-3292

Files

History

  • Receive Date: 25 July 2018
  • Revise Date: 13 November 2020
  • Accept Date: 05 July 2021

Share

How to cite

Statistics