Optimized deep networks structure to improve the accuracy of estimator algorithm in deep networks learning

Document Type : Article

Authors

1 Department of Computer, Kerman Branch, Islamic Azad University, Kerman, Iran

2 Department of Energy Management and Optimization, Institute of Science and High Technology and Environmental Sciences, Graduate University of Advanced Technology, Kerman, Iran

3 School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, UK

Abstract

An optimization algorithm based on training and learning is formed based on the process of training and learning in a class. A deep neural network is one of the types of feedforward neural networks whose connection pattern among its neurons is inspired by the visual cortex of animals' brain. The present study considers decreasing prediction error for the types of time series and the uncertainty in estimation parameters, improving the structure of the deep neural network and increasing response speed in the proposed neural network method; besides, the competitive performance and the collaboration among the neurons of deep neural network are also increased. Selected data is related to Qeshm weather (suitable weather conditions to study our purpose) prediction during 2016 onwards. In this study, for the purpose of analyzing the prediction issue of power consumption of domestic expenses in the indefinite and severe fluctuation mode, we decided to combine two methods of Long Short-Term Memory and Convolutional Neural Network. For the training of the deep network, the BP algorithm is used.

Keywords

Main Subjects

References

References:
1. Rao, R.V., Savsani, V.J., and Balic, J. "Teachinglearning-based optimization algorithm for unconstrained and constrained actual-parameter optimization problems", Engineering Optimization, 44(12), pp. 1447-1462 (2012). DOI: 10.1080/0305215X.2011.652103.
2. Schmidt-Hieber, A.J. "Nonparametric regression using deep neural networks with ReLU activation function", Annals of Statistics, 48(4), pp. 1875-1897 (2020). DOI: 10.1214/19-AOS1875.
3. Gorgoglione, A., Gioia, A., and Iacobellis, V. "A framework for assessing modeling performance and effects of rainfall-catchment-drainage characteristics on nutrient urban runoff in poorly gauged watersheds", Sustainability, 11(18), pp. 4933-4945 (2020). DOI: 10.3390/su11184933.
4. Niu, D. and Dai, S. "A short-term load forecasting model with a modified particle swarm optimization algorithm and least squares support vector machine based on the denoising method of empirical mode decomposition and grey relational analysis", Energies, 10(3), pp. 408-428 (2017). DOI: 10.3390/en10030408.
5. Wax, M. and Adler, A. "Detection of the number of signals by signal subspace matching. ieee trans", Signal Process, 69(1), pp. 973-985 (2021). DOI: 10.1109/TSP.2021.3053495.
6. Yadav, S., Wajid, M., and Usman, M., Support Vector Machine-Based Direction of Arrival Estimation with Uniform Linear Array, In book: Advances in Computational Intelligence Techniques Publisher: Springer, pp. 253-264 (2020). DOI: 10.1007/978-981-15-2620-6.
7. Choo, Y., Park, Y., and Seong, W. "Detection of direction-of-arrival in time domain using compressive time delay estimation with single and multiple measurements", Sensors (Basel), 20(18), pp. 5431-5442 (2020). DOI: 10.3390/s20185431.
8. Park, C. and Lee, D. "Classification of respiratory states using spectrogram with convolutional neural network", Applied Sciences, 12(4), pp. 1895-1906 (2022). DOI: 10.3390/app12041895.
9. Ketkar, N. and Moolayil, J., Convolutional Neural Networks. in Deep Learning With Python, Apress: Berkeley, CA, USA. pp. 197-242 (2021). DOI: 10.1007/978- 1-4842-5364-9 6.
10. Wei, Y. and Jiang, Z. "Estimating parameters of structural models using neural networks", USC Marshall School of Business Research Paper, 22(1), pp. 1-46 (2022). DOI: 10.2139/ssrn.3496098.
11. Wood, A., Wood, R., and Charnley, M. "Throughthe- wall radar detection using machine learning", Results in Applied Mathematics, 7(1), pp. 100106-100114 (2020). DOI: 10.1016/j.rinam.2020.100106.
12. Gregorczyk, M., Z_ orawski, P., Nowakowski, P., et al. "Sniffing detection based on network traffic probing and machine learning", In IEEE Access, 8(1), pp. 149255-149269 (2020). DOI: 10.1109/ACCESS.2020.3016076.
13. Cuntz, H. "Forest of synthetic pyramidal dendrites grown using cajal's laws of neuronal branching", PLoS Computational Biology, 6(8), ev06.i08-17. (2010). DOI: 10.1371/image.pcbi.v06.i08.
14. Bain, A., Mind and Body. the Theories of Their Relation, New York: d. Appleton and Company, James, the principles of psychology, New York: H. Holt and Company. (1873). URL: https://archive.org/details/ mindbodytheories00bain.
15. Brush, S.G. "History of the lenz-ising model", Reviews of Modern Physics, 39(4), pp. 883-893 (1967). DOI: 10.1103/RevModPhys.39.883.
16. Sherrington, C.S. "Experiments in examination of the peripheral distribution of the fibers of the posterior roots of some spinal nerves", Proceedings of the Royal Society of London, 190(1), pp. 45-186 (1989). DOI: 10.1098/rstb.1898.0002.
17. Li, Y., Lan, C., Xing, J., et al. "Online human action detection using joint classification-regression recurrent neural networks", In: 14th European Conference on Computer Vision - ECCV, Part VII, Springer (2017). DOI: 10.48550/arXiv.1604.05633.
18. Lahmiri, S. "Wavelet low- and high-frequency components as features for predicting stock prices with back propagation neural networks", Journal of King Saud University - Computer and Information Sciences, 26(2), pp. 218-227 (2014). DOI: 10.1016/j.jksuci.2013.12.001.
19. Ticknor, J.L. "A bayesian regularized artificial neural network for stock market forecasting", Expert Systems with Applications, 40(14), pp. 5501-5506 (2013). DOI: 10.1016/j.eswa.2013.04.013.
20. Kara, Y., Boyacioglu, M.A., and Baykan, O.K. "Predicting the direction of stock price index movement using artificial neural networks and support vector machines: The sample of the istanbul stock exchange", Journal of Expert Systems with Applications, 38(5), pp. 5311-5319 (2011). DOI: 10.1016/j.eswa.2010.10.027.
21. Zhang, H.M., Wang, Z.B., Wu, Z.H., et al. "Realtime through-the-wall radar imaging under unknown wall characteristics using the least-squares support vector machines based method", Journal of Applied Remote Sensing, 10(2), pp. 020501-0205011 (2021).DOI: 10.1117/1.JRS.10.020501.
22. Kose, U. and Arslan, A. "Forecasting chaotic time series via anfis supported by vortex optimization algorithm: Applications on electroencephalogram time series", Arab J Sci Eng., 42(1), pp. 3103-3114 (2017). DOI: 10.1007/s13369-016-2279-z.
23. Ma, X., Jin, Y., and Dong, Q. "A generalized dynamic fuzzy neural network based on singular spectrum analysis optimized by brain storm optimization for short-term wind speed forecasting", Applied Soft Computing, 54(1), pp. 296-312 (2017). DOI: 10.1016/j.asoc.2017.01.033.
24. Zhu, W., Lan, C., Xing, J., et al. "Co-occurrence feature learning for skeleton based action recognition using regularized deep lstm networks", AAAI Conference on Artificial Intelligence (2016) DOI: 10.48550/arXiv.1603.07772.
25. Zhang, S., Liu, X., and Xiao, J. "On geometric features for skeleton-based action recognition using multilayer lstm networks", 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (2017). DOI: 10.1109/TMM.2018.2802648.
26. Qiu, X., Ren, Y., Suganthan, P.N., et al. "Empirical mode decomposition based ensemble deep learning for load demand time series forecasting", Applied Soft Computing, 54(1), pp. 246-255 (2017). DOI: 10.1016/j.asoc.2017.01.015.
27. Tarigan, J., Diedan, R., and Suryana, Y. "Plate recognition using backpropagation neural network and genetic algorithm", Procedia Computer Science, 116(1), pp. 365-372 (2017). DOI: 10.1016/j.procs.2017.10.068.
28. Hannun, A.Y., Case, C., Casper, J., et al. "Deep speech: Scaling up end-to-end speech recognition", ArXiv. (2014). DOI: 10.48550/arXiv.1412.5567.
29. Sak, H., Senior, A.W., Rao, K., et al. "Learning acoustic frame labeling for speech recognition with recurrent neural networks", IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2015). DOI: 10.1109/ICASSP.2015.7178778.
Scientia Iranica
Volume 31, Issue 5 - Serial Number 5
Transactions on Computer Science & Engineering and Electrical Engineering (D)
March and April 2024
Pages 417-429

Files

History

  • Receive Date: 20 May 2023
  • Revise Date: 09 August 2023
  • Accept Date: 22 August 2023

Share

How to cite

Statistics