Joint distribution adaptation via feature and model matching

Document Type : Article

Authors

1 Faculty of IT & Computer Engineering, Urmia University of Technology, Urmia, Iran.

2 Faculty of IT & Computer Engineering, Urmia University of Technology, Urmia, Iran

Abstract

It is usually supposed that the training (source domain) and test (target domain) data follow a similar distribution and feature space in most pattern recognition tasks. However, in many real-world applications, particularly in visual recognition, this hypothesis has been frequently violated. This problem is known as domain shift problem. Domain adaptation and transfer learning are promising techniques to learn an effective and robust classifier to tackle shift problem. In this paper, we propose a novel scheme for domain adaptation, entitled as Joint Distribution Adaptation via Feature and Model Matching (JDAFMM), in which feature transform and model matching are jointly optimized. Due to joint optimization, we can have a robust model with feasible feature transformation and model parameter adaptation. By introducing regularization operated between the marginal and conditional distributions’ shifts across domains, we can successfully adapt data drift as much as possible along with empirical risk minimization and rate of consistency maximization between manifold and prediction function. We conduct extensive experiments to evaluate the performance of the proposed model against those of other machine learning and domain adaptation methods in three types of visual benchmark datasets. Our experiments illustrate that our JDAFMM significantly outperforms other baseline and state-of-the-art methods.

Keywords

Main Subjects

References

References:
1. Shi, Y. and Sha, F. "Information-theoretical learning of discriminative clusters for unsupervised domain adaptation", arXiv preprint arXiv:1206.6438 (2012).
2. Huang, J., Gretton, A., Borgwardt, K., Scholkopf, B., and Smola, A. "Correcting sample selection bias by unlabeled data", Neural Information Processing Systems (NIPS), pp. 601-608 (2007).
3. Pan, S.J., Tsang, I.W., Kwok, J.T., and Yang, Q. "Domain adaptation via transfer component analysis", IEEE Transactions on Neural Network, 22(2), pp. 199- 210 (2011).
4. Blitzer, J., Dredze, M., and Pereira, F. "Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification", ACL, 7, pp. 440-447 (2007).
5. Blitzer, J., McDonald, R., and Pereira, F. "Domain adaptation with structural correspondence learning", Conf. on Emp. Meth. in Natu. Lang. Proc., Sydney, Australia, pp. 120-128 (2006).
6. Duan, L., Xu, D., Tsang, I.W., and Luo, J. "Visual event recognition in videos by learning from web data", IEEE Trans. Pattern Anal. Mach. Intell, 34(9), pp. 1667-1680 (2012).
7. Jain, V. and Learned-Miller, E. "Online domain adaptation of a pre-trained cascade of classifiers", IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 577-584 (2011).
8. Tahmoresnezhad, J. and Hashemi, S. "Visual domain adaptation via transfer feature learning", Knowledge and Information Systems (KAIS), 50(2), pp. 588-605 (2017).
9. Gong, B., Shi, Y., Sha, F., and Grauman, K. "Geodesic flow kernel for unsupervised domain adaptation", IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2066-2073 (2012).
10. Tahmoresnezhad, J. and Hashemi, S. "Transductive transfer learning via maximum margin criterion", Scientia Iranica (SCI), 23(3), p. 1239 (2016).
11. Saenko, K., Kulis, B., Fritz, M., and Darrell, T. "Adapting visual category models to new domains", 11th European Conference on Computer Vision (ECCV), pp. 213-226 (2010).
12. Kumar, A., Saha, A., and Daume, H. "Coregularization based semi-supervised domain adaptation", Neural Information Processing Systems (NIPS), pp. 478-486 (2010).
13. Zhang, J., Li, W., and Ogunbona, P. "Joint geometrical and statistical alignment for visual domain adaptation", arXiv preprint arXiv:1705.05498 (2017).
14. Quanz, B., Huan, J., and Mishra, M. "Knowledge transfer with low-quality data: A feature extraction issue", IEEE Transactions on Knowledge and Data Engineering, 24(10), pp. 1789-1802 (2012).
15. Zhong, E., Fan, W., Peng, J., Zhang, K., Ren, J., Turaga, D., and Verscheure, O. "Cross domain distribution adaptation via kernel mapping", 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1027-1036 (2009).
16. Sun, Q., Chattopadhyay, R., Panchanathan, S., and Ye, J. "A two-stage weighting framework for multisource domain adaptation", Neural Information Processing Systems (NIPS), pp. 505-513 (2011).
17. Jolliffe, I.T. "Principal component analysis and factor analysis", PCA, pp. 150-166 (2002).
18. Gretton, A., Borgwardt, K.M., Rasch, M., Scholkopf, B., and Smola, A.J. "A kernel method for the twosample- problem", Neural Information Processing Systems (NIPS), pp. 513-520 (2007).
19. Aljundi, R., Emonet, R., Muselet, D., and Sebban, M. "Landmarks-based kernelized subspace alignment for unsupervised domain adaptation", IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 56-63 (2015).
20. Ishii, M., and Sato, A. "Joint optimization of feature transform and instance weighting for domain adaptation", International Joint Conference on Neural Networks (IJCNN), pp. 3793-3799 (2017).
21. Gong, B., Grauman, K., and Sha, F. "Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation", Int. Conf. on Mach. Learn., pp. 222-230 (2013).
22. Tahmoresnezhad, J. and Hashemi, S. "DiReT: An effective discriminative dimensionality reduction approach for multi-source transfer learning", Scientia Iranica (SCI), 24(3), pp. 1303-1311 (2017).
23. Liu, J., Li, J., and Lu, K. "Coupled local-global adaptation for multi-source transfer learning", Neurocomputing, 275, pp. 247-254 (2018).
24. Luo, L., Wang, X., Hu, S., Wang, C., Tang, Y., and Chen, L. "Close yet distinctive domain adaptation", arXiv preprint arXiv:1704.04235 (2017).
25. Luo, L., Wang, X., Hu, S., and Chen, L. "Robust data geometric structure aligned close yet discriminative domain adaptation", arXiv preprint arXiv:1705.08620 (2017).
26. Pan, S.J., Kwok, J.T., and Yang, Q. "Transfer learning via dimensionality reduction", 23d Conf. on Artif. Intel., Chicago, USA, pp. 677-682 (2008).
27. Long, M., Wang, J., Ding, G., Sun, J., and Yu, P.S. "Transfer feature learning with joint distribution adaptation", IEEE Int. Conf. on Computer Vision, pp. 2200-2207 (2013).
28. Aytar, Y. and Zisserman, A. "Tabula rasa: Model transfer for object category detection",Int. Conf. on Computer Vision (ICCV), pp. 2252-2259 (2011).
29. Gheisari, M. and Baghshah, M.S. "Joint predictive model and representation learning for visual domain adaptation", Engineering Applications of Artif. Intel., 58, pp. 157-170 (2017).
30. Yang, J., Yan, R., and Hauptmann, A.G. "Adapting SVM classifiers to data with shifted distributions", 7th IEEE Int. Conf. on Data Mining Workshops, pp. 69-76 (2007).
31. Bruzzone, L. and Marconcini, M. "Domain adaptation problems: A DASVM classification technique and a circular validation strategy", IEEE Trans. Pattern Anal. Mach. Intell, 32(5), pp. 770-787 (2010).
32. Long, M., Wang, J., Ding, G., Pan, S.J., and Yu, P.S. "Adaptation regularization: A general framework for transfer learning", IEEE Transactions on Knowledge and Data Engineering, 26(5), pp. 1076-1089 (2014).
33. Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.P., Scholkopf, B., and Smola, A.J. "Integrating structured biological data by kernel maximum mean discrepancy", Bioinformatics, 22(14), pp. e49-e57 (2006).
34. Belkin, M., Niyogi, P., and Sindhwani, V. "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples", J. Mach. Learn. Res, 7(Nov), pp. 2399-2434 (2006).
35. Von Luxburg, U. "A tutorial on spectral clustering", Statistics and Computing (SC), 17(4), pp. 395-416 (2007).
36. Scholkopf, B., Herbrich, R., and Smola, A. "A generalized representer theorem", Computational Learning Theory, 11, pp. 416-426 (2001).
37. Long, M., Wang, J., Sun, J., and Philip, S.Y. "Domain invariant transfer kernel learning", IEEE T KNOWL DATA EN, 27(6), pp. 1519-1532 (2015).
38. Long, M., Wang, J., Ding, G., Sun, J., and Yu, P.S. "Transfer joint matching for unsupervised domain adaptation", IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 1410-1417 (2014).
Scientia Iranica
Volume 26, Special Issue on machine learning, data analytics, and advanced optimization techniques...
Transactions on Computer Science & Engineering and Electrical Engineering (D)
November and December 2019
Pages 3515-3539

Files

History

  • Receive Date: 31 October 2017
  • Revise Date: 25 May 2018
  • Accept Date: 08 December 2018

Share

How to cite

Statistics