Scientia Iranica

Scientia Iranica

Adaptive inverse deep reinforcement Lyapunov learning control for a floating wind turbine

Document Type : Research Article

Authors
Hadi Mohammadian KhalafAnsar
Jafar Keighobadi
Faculty of Mechanical Engineering, University of Tabriz, East Azerbaijan, Iran.
10.24200/sci.2023.61871.7532
Abstract
Offshore Floating Wind Turbines (FWT) decrease climate change adversial effects without occupying significant land and harvesting fields. Owing to the earth planet unexpected climate, online adaptive feedback control of FWTs will be effective in the sense of optimal and uniform energy capture. In this paper, a Deep Reinforcement Learning (DRL)-based control system is proposed to offset both the disturbance and noise effects. Large variations of wind and water waves generate enormous information give rise to convergent learning of deep neural networks model of the wind turbine. As a result of the disturbance and wind sudden variations, an adaptive inverse control equipped with DRL could easily cope with the inherent drawback of DRL i.e., tracking error. Furthermore, received rewards in the DRL algorithm are passed through the newly designed training algorithm to predict control actions such that the loss function is decreased. The attenuation of disturbance and noise on the tracking performance of closed-loop FWT is clarified through software implementation tests while the weight’s convergency and update rules are proved by the direct Lyapunov theorem.
Keywords
Deep Reinforcement Learning (DRL)
Artificial Intelligence (AI)
Adaptive inverse control
Deep deterministic policy gradient
Floating Wind Turbine (FWT)
Subjects
References
1. Widrow, B., Duvall, K.M., Gooch, R.P., et al. “Signal cancellation phenomena in adaptive antennas: Cases and cures”, IEEE Trans, Antennas Propag., 30(3), pp. 469-478 (1982). https://doi.org/10.1109/TAP.1982.1142804
2. Jim, C.W. “A comparison of two LMS constrained optimal array structures”, Proc. IEEE, 65(2), pp. 1730-1731 (1977). https://doi.org/10.1109/PROC.1977.10820
3. Griffiths, L.J. and Jim, C.W. “An alternative approach to linearly constrained adaptive beam forming”, IEEE Trans, Antenna Propag., 30(1), pp. 27-34 (1982). https://doi.org/10.1109/TAP.1982.1142739
4. Usman, M., Luna, A.A., and Elahi, I. “A stable and adaptive pole-zero filter”, IEEE Transactions on Consumer Electronics, 49(4), pp. 1279-1285(20032). https://doi.org/10.1109/TCE.2003.1261230
5. Hesse, M., Timmermann, J., Hullermeier, E., et al. “A reinforcement learning strategy for the swing-up of the double pendulum on a cart”, Procedia Manuf, 24, pp. 15-20 (2018). https://doi.org/10.1016/j.promfg.2018.06.004
6. Manrique, C., Pappalardo, C.M., and Guida, D. “A model validating technique for the kinematic study of two-wheeled vehicles”, Springer International Publishing, Odessa, Ukraine, pp. 549-558 (2020). https://doi.org/10.1007/978-3-030-40724-7_56
7. Pappalardo, C.M., De Simone, M.C., and Guida, D. “Multi body modeling and nonlinear control of the pantograph/catenary system”, Arch. Appl., Mech., 89, pp. 1589-1626 (2019). https://doi.org/10.1007/s00419-019-01530-3
8. Pappalardo, C. and Guida, D. “Forward and inverse dynamics of a unicycle-like mobile robot”, Machines, 7(1), 5 (2019). https://doi.org/10.3390/machines7010005
9. Villecco, F. and Pellegrino, A. “Evaluation of uncertainties in the design process of complex mechanical systems”, Entropy, 19(9), 475 (2017). https://doi.org/10.3390/e19090475
10. Villecco, F., and Pellegrino, A., “Entropic measure of epistemic uncertainties in multi body system models by axiomatic design”, Entropy, 19(7), 291 (2017). https://doi.org/10.3390/e19070291
11. Hu, D., Pei, Z., and Tang, Z., “Single-parameter-tuned attitude control for quad rotor with unknown disturbance”, Appl., Sci., 10(16), 5564 (2020). https://doi.org/10.3390/app10165564
12. Talamini, J., Bartoli, A., De Lorenzo, A.D., et al. “On the impact of the rules on autonomous drive learning”, Appl. Sci., 10(7), 2394 (2020). https://doi.org/10.3390/app10072394
13. Sharifzadeh, S., Chiotellis, I., Triebel, R., et al. “Learning to drive using inverse reinforcement learning and deep Q-networks” (2016). https://doi.org/10.48550/arXiv.1612.03653
14. Cho, N.J., Lee, S.H., Kim, J.B., et al. “Learning, improving, and generalizing motor skills for the peg-in-hole tasks based on imitation learning and self-learning”, Appl. Sci., 10(8), 2719 (2020). https://doi.org/10.3390/app10082719
15. Zhang, H., Qu, C., Zhang, J., et al. “Self-adaptive priority correction for prioritized experience replay”, Appl. Sci., 10(19), 6925 (2020). https://doi.org/10.3390/app10196925
16. Hong, D., Kim, M., and Park, S. “Study on reinforcement learning-based missile guidance law”, Appl. Sci., 10(16), 6567 (2020). https://doi.org/10.3390/app10186567
17. Rivera, Z.B., De Simone, M.C., and Guida, D., “Unmanned ground vehicle modelling in Gazebo/ROS-Based environments”, Machines, 7(2), 42 (2019). https://doi.org/10.3390/machines7020042
18. Kaveh, A. and S. Sabeti. “Optimal design of monopile offshore wind turbine structures using CBO, ECBO, and VPS algorithms”, Scientia Iranica, 26(3), pp. 1232-1248 (2019). https://doi.org/10.24200/sci.2018.20090
19. Mohammadi Ajamloo, A., Abbaszadeh, K., and Nasiri-Zarandi., R., “A novel transverse flux permanent magnet generator for small-scale direct drive wind turbine application: Design and analysis”, Scientia Iranica, 28(6), pp. 3363-3378 (2021). https://doi.org/10.24200/sci.2019.53145.3078
20. Maftouni, N. and Taghaddosi, M. “A CFD study of a flanged shrouded wind turbine: Effects of different flange surface types on output power”, Scientia Iranica, 29(1), pp. 101-108 (2022). https://doi.org/10.24200/sci.2021.57513.5278
21. Shamsnia, A. and Parniani, M. “A comparative analysis of the new excitation controlled synchronous generator-based wind turbine”, Scientia Iranica, 29(1), pp. 151-167 (2022). https://doi.org/10.24200/sci.2019.51354.2132
22. Abedini, M., Vahabzadeh, T., Ahmadi, S.-A., et al. “Smart microgrid educational laboratory: An integrated electric and communications infrastructure platform”, Scientia Iranica, 29(5), pp. 2552-2565 (2022). https://doi.org/10.24200/sci.2020.55942.4483
23. Zhang, J., Zhao, X., and Wei, X. “Reinforcement learning-based structural control of floating wind turbines”, In IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(3), pp. 1603-1613 (2022). https://doi.org/10.1109/TSMC.2020.3032622
24. Tang, B., Chen, Y., Chen, Q., et al. “Research on short-term wind power forecasting by data mining on historical wind resource”, Applied Sciences, 10(4), 1295 (2020). https://doi.org/10.3390/app10041295
25. Yao, W., Huang, P., and Jia, Z., “Multidimensional LSTM networks to predict wind speed”, In 2018 37th Chinese Control Conference (CCC), pp. 7493-7497 (2018). https://doi.org/10.23919/ChiCC.2018.8484017
26. Gu, C. and Li, H., “Review on deep learning research and applications in wind and wave energy”, Energies, 15(4), 1510 (2022). https://doi.org/10.3390/en15041510
27. Liu, H., Mi, X., and Li, Y. “Smart multi-step deep learning model for wind speed forecasting based on variational mode decomposition, singular spectrum analysis, LSTM network and ELM”, Energy Conversion and Management, 159, pp. 54-64 (2018). https://doi.org/10.1016/j.enconman.2018.01.010
28. Sierra-García, J.E. and Santos, M. “Switched learning adaptive neuro-control strategy”, Neuro computing, 452, pp. 450-464 (2021). https://doi.org/10.1016/j.neucom.2019.12.139
29. Márquez, F.P.G. and Chacón, A.M.P. “A review of non-destructive testing on wind turbines blades”, Renewable Energy, 161, pp. 998-1010 (2020). https://doi.org/10.1016/j.renene.2020.07.145
30. Keighobadi, J., KhalafAnsar, H.M., and Naseradinmousavi, P. “Adaptive neural dynamic surface control for uniform energy exploitation of floating wind turbine”, Applied Energy, 316, 119132 (2022). https://doi.org/10.1016/j.apenergy.2022.119132
31. Manrique Escobar, C.A., Pappalardo, C.M., and Guida, G. “A parametric study of a deep reinforcement learning control system applied to the swing-up problem of the cart-pole”, Applied Sciences, 10(24), 9013 (2020). https://doi.org/10.3390/app10249013
32. Van der Vaart, H.R. and Yen, E.H. “Weak sufficient conditions for fatou's lemma and lebesgue's dominated convergence theorem”, Mathematics Magazine, 41(3), pp. 109-117 (1968). https://doi.org/10.1080/0025570X.1968.11975853
33. François-Lavet, V., Henderson, P., Islam, R., et al. “An introduction to deep reinforcement learning”, Found. Trends Mach, Learn, 11(3-4), pp. 219-354 (2018). https://doi.org/10.1561/220000007
34. Yang, Y., Li, X., and Zhang, L. “Task-specific pre-learning to improve the convergence of reinforcement learning based on a deep neural network”, In Proceedings of the 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, China, pp. 2209-2214 (2016). https://doi.org/10.1109/WCICA.2016.7578787
35. Zagal, J.C., Ruiz-del-Solar, J., and Vallejos, P. “Back to reality: Crossing the reality gap in evolutionary robotics”, IFAC Proc., 37(8), pp. 834-839 (2004). https://doi.org/10.1016/S1474-6670(17)32084-0
36. Bekar, C., Yuksek, B., and Inalhan, G. “High fidelity progressive reinforcement learning for agile maneuvering UAVs”, In Proceedings of the AIAA Scitech 2020 Forum; American Institute of Aeronautics and Astronautics, Orlando, FL. USA, pp. 1-12 (2020). https://doi.org/10.2514/6.2020-0898
37. Al-Araji, A.S. “An adaptive swing-up sliding mode controller design for a real inverted pendulum system based on Culture-Bees algorithm”, Eur. J., Control, 45, pp. 45-56 (2019). https://doi.org/10.1016/j.ejcon.2018.12.001
38. Rauh, A., Grigoryev, V., Aschemann, H., et al. “Incremental gain scheduling and sensitivity-based control for under actuated ships”, In Proc. of IFAC Conference on Control Applications in Marine Systems, CAMS 2010, Rostock-Warnemunde, Germany, 43(20), pp. 161-166 (2010). https://doi.org/10.3182/20100915-3-DE-3008.00055
39. Rauh, A. and Aschemann, H. “Sensitivity-based feed forward and feedback control using algorithmic differentiation”, In Proc. of IEEE Intl. Conference on Methods and Models in Automation and Robotics MMAR, 2010, Miedzyzdroje, Poland, pp. 7-12 (2010). https://doi.org/10.1109/MMAR.2010.5587271
Scientia Iranica
Volume 32, Issue 17
Transactions on Computer Science & Engineering and Electrical Engineering
September and October 2025 Article ID:7532

  • Receive Date 01 February 2023
  • Revise Date 30 May 2023
  • Accept Date 28 June 2023