Time series prediction with a hybrid system formed by artificial neural network and cognitive development optimization algorithm

Document Type : Article

Authors

1 Computer Sciences Application and Research Center, Usak University, Usak, Turkey.

2 Department of Computer Engineering, Konya Food and Agriculture University, Konya, Turkey.

Abstract

Time series prediction is a remarkable research interest, which is widely followed by scientists / researchers. Because many fields include analyzing processes over such time series, different kinds of approaches, methods, and techniques are often employed in order to achieve alternative prediction ways. It seems that Artificial Intelligence oriented solutions have strong potential on providing effective and accurate prediction approaches in even most complicated time series structures. In the sense of the explanations, this study aims to introduce an alternative, Artificial Intelligence based approach of Artificial Neural Networks, and Cognitive Development Optimization Algorithm, a recent intelligent optimization technique introduced by the authors. Here, it has been aimed to predict different kinds of time series, by using the introduced system / approach. In this way it has been possible to discuss about application potential of the hybrid system and report findings related to its success on prediction. The authors believe that the study has been a good chance to support the literature with an alternative solution approach and see potential of a newly developed, Artificial Intelligence oriented optimization algorithm on different applications.

Keywords

Main Subjects


1. Douglas, A.I., Williams, G.M., Samuel, A.W., and  Carol, A.W., Basic Statistics for Business and Economics,  3/e., McGraw-Hill (2009).  2. Esling, P. and Agon, C. Time-series data mining",  ACM Computing Surveys (CSUR), 45(1), p. 12  (2012).  3. NIST SEMATECH, Introduction to Time Series  Analysis, In Engineering Statistics Handbook,  http://www.itl.nist.gov/div898/handbook/pmc/  section 4/pmc4. htm Online. Retrieved on 10th July  (2016).  4. Penn State Eberly Collage of Science, Overview  of time series characteristics", STAT-510, App.  Time Series Analysis, https://onlinecourses. science.  psu.edu/stat510/node/47, Online. Retrieved on  10th July (2016).  5. Gromov, G.A. and Shulga, A.N. Chaotic time series  prediction with employment of ant colony optimization",  Expert Systems with App., 39, pp. 8474-8478  (2012).  6. Kose, U. and Arslan, A. Realizing an optimization  approach inspired from Piaget's theory on cognitive  development", Broad Res. in arti_cial intelligence  and Neuroscience, 6(1-2), pp. 14-21 (2015).  7. Piaget, J. Part I: Cognitive development in children:  Piaget development and learning", Journal of Res. in  Science Teaching, 2(3), pp. 176-186 (1964).  8. Piaget, J., Main Trends in Psychology, London:  George Allen and Unwin (1973).  9. Singer, D.G., Revenson, T.A.A., and Piaget, P.,  How a Child Thinks. International Universities Press,  Inc., 59 Boston Post Road, Madison, CT 06443-1524  (1997).  10. Kose, U. and Arslan, A. Forecasting chaotic time  series via an_s supported by vortex optimization algorithm:  Applications on electroencephalogram time  series", Arabian Journal for Science and Eng., 42(8),  pp. 3103-3114 (2017).  11. Gan, M., Peng, H., Peng, X., Chen, X., and  Inoussa, G. A locally linear RBF network-based  state-dependent AR model for nonlinear time series  modeling", Information Sciences, 180, pp. 4370-4383  (2010).  12. Wong, W.K., Xia, M., and Chu, W.C. Adaptive  neural network model for time-series forecasting",  European Journal of Operational Research, 207, pp.  807-816 (2010).  13. Gentili, P.L., Gotoda, H., Dolnik, M., and Epstein,  I.R. Analysis and prediction of aperiodic hydrodynamic  oscillatory time series by feed-forward neural  networks, fuzzy logic, and a local nonlinear predictor",  Chaos: An Interdiscip. Journal of Nonlinear  Science, 25(1), 013104 (2015).  14. Chen, D. and Han, W. Prediction of multivariate  chaotic time series via radial basis function neural  network", Complexity, 18(4), pp. 55-66 (2013).  15. Wu, X., Li, C., Wang, Y., Zhu, Z., and Liu,  W. Nonlinear time series prediction using iterated  extended Kalman _lter trained single multiplicative  neuron model", Journal of Information and Comp.  Science, 10, pp. 385-393 (2013).  16. Yadav, R.N., Kalra, P.K., and John, J. Time series  prediction with single multiplicative neuron model",  Applied Soft Computing, 7, pp. 1157-1163 (2007).  17. Zhao, L. and Yang, Y. PSO-based single multiplicative  neuron model for time series prediction", Expert  Systems with App., 36, pp. 2805-2812 (2009).  18. Yao, J. and Liu, W. Nonlinear time series prediction  of atmospheric visibility in Shanghai", In Time Series  Analysis, Modeling and Applications, Intelligent Systems  Reference Library, 47, W. Pedrycz and S.-M.  Chen, Eds., Springer-Verlag (2013)  19. Unler, A. Improvement of energy demand forecasts  using swarm intelligence: The case of Turkey with  projections to 2025", Energy Policy, 36, pp. 1937-  1944 (2008).  20. Zhao, L. and Yang, Y. PSO-based single multiplicative  neuron model for time series prediction", Expert  Systems with Apps, 36, pp. 2805-2812 (2009).  21. Weng, S.S. and Liu, Y.H. Mining time series data  for segmentation by using ant colony optimization",  European Journal of Operational Research, 173, pp.  921-937 (2006).  22. Toskari, M.D. Estimating the net electricity energy  generation and demand using the ant colony optimization  approach", Energy Policy, 37, pp. 1181-  1187 (2009).  23. Hong, W.C. Application of chaotic ant swarm optimization  in electric load forecasting", Energy Policy,  38, pp. 5830-5839 (2010).  24. Niu, D., Wang, Y., and Wu, D.D. Power load  forecasting using support vector machine and ant  colony optimization", Expert Systems with App., 37,  pp. 2531-2539 (2010).  25. Yeh, W.-C. New parameter-free simpli_ed swarm  optimization for arti_cial neural network training and  its application in the prediction of time series", IEEE  Trans. on Neural Networks and Learning Systems, 24,  pp. 661-665 (2013).  26. Nourani, V. and Andalib, G. Wavelet based arti_cial  intelligence approaches for prediction of hydrological  time series", Australasian Conference on Arti_cial  Life and Comp. Intelligence, pp. 422-435, Newcastle,  NSW, Australia (2015).  27. Bontempi, G., Taieb, S.B., and Le Borgne, Y.-A.  Machine learning strategies for time series forecasting",  In Business Intelligence (Lecture Notes in Business  Information Processing), -138, M.-A. Aufaure  and E. Zimanyi, Eds., Springer-Verlag (2013).  28. Hu, Y.X. and Zhang, H.T. Prediction of the chaotic  time series based on chaotic simulated annealing and  support vector machine", Int. Conference on Solid  State Devices and Materials Science, pp. 506-512.  Macao, China (2012).  29. Liu, P. and Yao, J.A. Application of least square  support vector machine based on particle swarm optimization  to chaotic time series prediction", IEEE Int.  Conference on Intelligent Computing and Intelligent  Systems, pp. 458-462. Shanghai, China (2009).  30. Quian, J.S., Cheng, J., and Guo, Y.N. A novel  multiple support vector machines architecture for  chaotic time series prediction", Advances in Natural  Computation, Lecture Notes in C.S., 4221, pp. 147-  156 (2006).  31. Yang, Z.H.O., Wang, Y.S., Li, D.D., and Wang, C.J.  Predict the time series of the parameter-varying  chaotic system based on reduced recursive lease  square support vector machine", IEEE Int. Conference  on Arti_cial Intelligence and Comp. Intelligence,  pp. 29-34, Shanghai, China (2009).  32. Zhang, J.S., Dang, J.L., and Li, H.C. Local support  vector machine prediction of spatiotemporal chaotic  time series", Acta Physica Sinica, 56, pp. 67-77  (2007).  33. Farooq, T., Guergachi, A., and Krishnan, S. Chaotic  time series prediction using knowledge based Green's  kernel and least-squares support vector machines",  IEEE Int. Conference on Systems, Man and Cybernetics,  pp. 2669-2674, Montreal, Cook Islands (2007).  34. Shi, Z.W. and Han, M. Support vector echo-state  machine for chaotic time-series prediction", IEEE  Trans. on Neural Networks, 18, pp. 359-372 (2007).  35. Li, H.T. and Zhang, X.F. Precipitation time series  predicting of the chaotic characters using support  vector machines", Int. Conference on Information  Management, Innovation Management and Indus.  Eng., pp. 407-410, Xian, China (2009).  36. Zhu, C.H., Li, L.L., Li, J.H., and Gao, J.S. Shortterm  wind speed forecasting by using chaotic theory  and SVM", Applied Mechanics and Materials, 300-  301, pp. 842-847 (2013).  37. Ren, C.-X., Wang, C.-B., Yin, C.-C., Chen, M., and  Shan, X. The prediction of short-term tra_c ow  based on the niche genetic algorithm and BP neural  network", 2012 Int. Conference on Information  Technology and Software Engineering, pp. 775-781,  Beijing, China (2013).  38. Ding, C., Wang, W., Wang, X., and Baumann, M.  A neural network model for driver's lane-changing  trajectory prediction in urban tra_c ow", Mathematical  Problems in Engineering (Online) (2013)  DOI: 10.1155/2013/967358  39. Yin, H., Wong, S.C., Xu, J., and Wong, C.K. Urban  tra_c ow prediction using a fuzzy-neural approach",  Transportation Research Part-C: Emerging Technologies,  10, pp. 85-98 (2002).  40. Dunne, S. and Ghosh, B. Weather adaptive tra_c  prediction using neurowavelet models", IEEE Trans.  on Intelligent Transportation Systems, 14, pp. 370-  379 (2013).  41. Pulido, M., Melin, P., and Castillo, O. Particle  swarm optimization of ensemble neural networks with  fuzzy aggregation for time series prediction of the  Mexican stock exchange", Information Sciences, 280,  pp. 188-204 (2014).  42. Huang, D.Z., Gong, R.X., and Gong, S. Prediction  of wind power by chaos and BP arti_cial neural networks  approach based on genetic algorithm", Journal  of Electrical Eng. and Tech., 10(1), pp. 41-46, (2015).  43. Jiang, P., Qin, S., Wu, J., and Sun, B. Time series  analysis and forecasting for wind speeds using support  vector regression coupled with arti_cial intelligent  algorithms", Mathematical Prob. in Eng., Article ID  939305 (2015).  44. Doucoure, B., Agbossou, K., and Cardenas, A. Time  series prediction using arti_cial wavelet neural network  and multi-resolution analysis: Application to  wind speed data", Renewable Energy, 92, pp. 202-  211 (2016).  45. Chandra, R. Competition and collaboration in cooperative  coevolution of Elman recurrent neural networks  for time-series prediction", IEEE Trans. on    Neural Networks and Learning Systems, 26(12), pp.  3123-3136 (2015).  46. Chai, S.H. and Lim, J.S. Forecasting business cycle  with chaotic time series based on neural network  with weighted fuzzy membership functions", Chaos,  Solitons and Fractals, 90, pp. 118-126 (2016).  47. Seo, Y., Kim, S., Kisi, O., and Singh, V.P. Daily  water level forecasting using wavelet decomposition  and arti_cial intelligence techniques", Journal of  Hydrology, 520, pp. 224-243 (2015).  48. Marzban, F., Ayanzadeh, R., and Marzban, P. Discrete  time dynamic neural networks for predicting  chaotic time series", Journal of Arti_cial Intelligence,  7(1), p. 24 (2014).  49. Okkan, U. Wavelet neural network model for reservoir  inow prediction", Scientia Iranica, 19(6), pp.  1445-1455 (2012).  50. Zhou, T., Gao, S., Wang, J., Chu, C., Todo, Y.,  and Tang, Z. Financial time series prediction using a  dendritic neuron model", Knowledge-Based Systems,  105, pp. 214-224 (2016).  51. Wang, L., Zou, F., Hei, X., Yang, D., Chen, D.,  Jiang, Q., and Cao, Z. A hybridization of teachinglearning-  based optimization and di_erential evolution  for chaotic time series prediction", Neural Comp. and  App., 25(6), pp. 1407-1422 (2014).  52. Heydari, G., Vali, M., and Gharaveisi, A.A. Chaotic  time series prediction via arti_cial neural square fuzzy  inference system", Expert Systems with App., 55, pp.  461-468 (2016).  53. Wang, L., Zeng, Y., and Chen, T. Back propagation  neural network with adaptive di_erential evolution  algorithm for time series forecasting", Expert Systems  with App., 42(2), pp. 855-863 (2015).  54. Catalao, J.P.S., Pousinho, H.M.I., and Mendes,  V.M.F. Hybrid wavelet-PSO-ANFIS approach for  short-term electricity prices forecasting", IEEE  Trans. on Power Systems, 26(1), pp. 137-144 (2011).  55. Patra, A., Das, S., Mishra, S.N., and Senapati,  M.R. An adaptive local linear optimized radial basis  functional neural network model for _nancial time  series prediction", Neural Comp. and App., 28(1), pp.  101-110 (2017).  56. Ravi, V., Pradeepkumar, D., and Deb, K. Financial  time series prediction using hybrids of chaos  theory, multi-layer perceptron and multi-objective  evolutionary algorithms", Swarm and Evolutionary  Computation, 36, pp. 136-149 (2017).  57. M_endez, E., Lugo, O., and Melin, P. A competitive  modular neural network for long-term time series  forecasting", In Nature-Inspired Design of Hybrid  Intelligent Systems, pp. 243-254, Springer Int. Publishing  (2017).  58. Carpenter, G.A. Neural network models for pattern  recognition and associative memory", Neural Networks,  2(4), pp. 243-257 (1989).  59. Cochocki, A. and Unbehauen, R., Neural Networks  for Optimization and Signal Processing, John Wiley  & Sons, Inc. (1993).  60. Miller, W.T., Sutton, R.S., and Werbos, P.J., Neural  Networks for Control, MIT Press (1995).  61. Ripley, B.D. Neural networks and related methods  for classi_cation", Journal of the Royal Statistical Society,  Series-B (Methodological), pp. 409-456 (1994).  62. Basheer, I.A. and Hajmeer, M. Arti_cial neural  networks: fundamentals, computing, design and application",  Journal of Microbiological Methods, 43,  pp. 3-31 (2000).  63. Badri, A., Ameli, Z., and Birjandi, A.M. Application  of arti_cial neural networks and fuzzy logic methods  for short term load forecasting", Energy Procedia, 14,  pp. 1883-1888 (2012).  64. Ghorbanian, J., Ahmadi, M., and Soltani, R. Design  predictive tool and optimization of journal bearing  using neural network model and multi-objective genetic  algorithm", Scientia Iranica, 18(5), pp. 1095-  1105 (2011).  65. Gholizadeh, S. and Seyedpoor, S.M. Shape optimization  of arch dams by metaheuristics and neural  networks for frequency constraints", Scientia Iranica,  18(5), pp. 1020-1027 (2011).  66. Firouzi, A. and Rahai, A. An integrated ANNGA  for reliability based inspection of concrete bridge  decks considering extent of corrosion-induced cracks  and life cycle costs", Scientia Iranica, 19(4), pp. 974-  981 (2012).  67. Shahreza, M.L., Moazzami, D., Moshiri, B., and  Delavar, M.R. Anomaly detection using a selforganizing  map and particle swarm optimization",  Scientia Iranica, 18(6), pp. 1460-1468 (2011).  68. Ruck, D.W., Rogers, S.K., and Kabrisky, M. Feature  selection using a multilayer perceptron", Journal of  Neural Network Comp., 2(2), pp. 40-48 (1990).  69. Kose, U. and Arslan, A. Optimization of selflearning  in computer engineering courses: An intelligent  software system supported by arti_cial neural  network and vortex optimization algorithm", Computer  Applications in Engineering Education, 25(1),  pp. 142-156 (2017).  70. McCulloch, W.S. and Pitts, W. A logical calculus  of the ideas immanent in nervous activity", Bulletin  of Mathematical Biology, January, 52, pp. 99-115  (1943). Reprint: Bulletin of Mathematical Biophysics,  5, pp. 115-133 (1990).  71. Anderson, D. and McNeill, G. Arti_cial neural networks  technology", A DACS state-of-the-art report.  Kaman Sciences Corporation, 258, PP. 13502-13462  (1992).  72. Ugur, A. and Kinaci, A.C. A web-based tool for  teaching neural network concepts", Computer App.  in Engineering Educ., 18(3), pp. 449-457 (2010).  73. Yegnanarayana, B., Arti_cial Neural Networks, PHI  Learning Pvt. Ltd (2009).  74. Demir, A. and Kose, U. Solving optimization problems  via vortex optimization algorithm and cognitive  development optimization algorithm", Broad  Research in Arti_cial Intelligence and Neuroscience,  7(4), pp. 23-42 (2016).  75. Kose, U. and Arslan, A. Intelligent e-learning system  for improving students' academic achievements  in computer programming courses", Int. Journal of  Engineering Educ., 32(1), pp. 185-198 (2016).  76. Blum, C. and Li, X. Swarm intelligence in optimization",  In Swarm Intelligence, C. Blum and D. Merkle,  Eds., Springer Berlin Heidelberg (2008).  77. Engelbrecht, A.P., Fundamentals of Computational  Swarm Intelligence, John Wiley & Sons (2006).  78. Bonabeau, E., Dorigo, M., and Theraulaz, G., Swarm  Intelligence: From Natural to Arti_cial Systems (No.  1), Oxford Univ. Press (1999).  79. Panigrahi, B.K., Shi, Y., and Lim, M.H. (Eds.).,  Handbook of Swarm Intelligence: Concepts, Principles  and Applications, 8, Springer Sci. & Business  Media (2011).  80. Fukuyama, Y. Fundamentals of particle swarm optimization  techniques", In Modern Heuristic Optimization  Techniques: Theory and Applications to Power  Systems, K.Y. Lee and M.A. El-Sharkawi, Eds., John  Wiley & Sons, Hoboken, N.J., USA (2008).  81. Bonabeau, E., Dorigo, M., and Theraulaz, G. Inspiration  for optimization from social insect behaviour",  Nature, 406(6791), pp. 39-42 (2000).  82. Kennedy, J. Particle swarm optimization", In Encyclopedia  of Machine Learning, C. Sammut and G.I.  Webb, Eds., Springer-US (2011).  83. Dorigo, M. and Blum, C. Ant colony optimization  theory: A survey", Theoretical Computer Science,  344(2), pp. 243-278 (2005).  84. Karaboga, D., Arti_cial Intelligence Optimization  Algorithms, Nobel Publishing, Turkey, ISBN 975-  6574 (2004).  85. DataMarket DataMarket- _nd, understand and  share data", https://datamarket.com/ Online. Retrieved  on 12th July (2016).  86. Scholarpedia Mackey-glass equation", http://  www.scholarpedia.org/article/Mackey-Glassequation  Online. Retrieved on 12th July (2016).  87. OTexts.org. Evaluating forecast accuracy",  https://www.otexts.org/fpp/2/5 Online. Retrieved  on 16th July (2016).  88. Eberhart, R.C. Kennedy, J. A new optimizer using  particle swarm theory", Sixth International Symposium  on Micro Machine and Human Science, pp. 39-  43 (1995).  89. Kennedy, J. The particle swarm: social adaptation  of knowledge", IEEE Int. Conference on Evol. Comp.,  pp. 303-308, IEEE (1997).  90. Yang, X.S. and Deb, S. Cuckoo search via L_evy  ights", World Congress on Nature & Biologically  Ins. Comp., pp. 210-214, IEEE (2009).  91. Yang, X.S. and Deb, S. Cuckoo search: recent  advances and applications", Neural Comp. and App.,  24(1), pp. 169-174 (2014).  92. Yang, X.S., Nature-Inspired Metaheuristic Algorithms,  Luniver Press (2010).  93. Yang, X.S. Firey algorithms for multimodal optimization",  In Stochastic Algorithms: Foundations  and Applications, O. Watanabe and T. Zeugmann,  Eds., Springer Berlin Heidelberg (2009).  94. Yang, X.S. A new metaheuristic bat-inspired algorithm",  In Nature Inspired Cooperative Strategies for  Optimization, (NICSO 2010), J.R. Gonz_elez, D.A.,  Pelta, C., Cruz, G. Terrazas, and N. Krasnogor, Eds.,  Springer Berlin Heidelberg (2010).  95. Yang, X.S. and Hossein Gandomi, A. Bat algorithm:  a novel approach for global engineering optimization",  Engineering Computations, 29(5), pp. 464-483  (2012).  96. Dasgupta, S. and Osogami, T. Nonlinear Dynamic  Boltzmann Machines for Time-Series Prediction", In  AAAI, 31(12), pp. 1833-1839 (2017).  97. Kim, K.J. Financial time series forecasting using  support vector machines", Neurocomputing, 55(1),  pp. 307-319 (2003).  98. Hassan, M.R. and Nath, B. Stock market forecasting  using hidden Markov model: a new approach. In intelligent  systems design and applications", ISDA'05.  Proceedings. 5th Int. Conference on IEEE, pp. 192-  196 (2005).  99. Brahim-Belhouari, S. and Bermak, A. Gaussian  process for nonstationary time series prediction",  Comp. Statistics & Data Analysis, 47(4), pp. 705-712  (2004).  100. Giarratano, J.C. and Riley, G., Expert Systems, PWS  Publishing Co. (1998).  101. Turban, E. and Frenzel, L.E., Expert Systems and Applied  Arti_cial Intelligence, Prentice Hall Professional  Tech. Reference (1992).  102. David, J.M., Krivine, J.P., and Simmons, R. Eds.,  Second Generation Expert Systems, Springer Sci.&  Business Media (2012).