A POMDP framework to find optimal policy in sustainable maintenance

Ghandali, R.; Abooie, M.H.; Fallahnezhad, M.S.

doi:10.24200/sci.2018.5490.1306

A POMDP framework to find optimal policy in sustainable maintenance

Document Type : Research Article

Authors

Department of Industrial Engineering, Yazd University, Yazd, Iran.

10.24200/sci.2018.5490.1306

Abstract

The increasing importance of these two subjects, maintenance and cleaner environment, beside the relations between them, encourages us to investigate a mathematical Markovian model for Condition Based Maintenance problem while considering environmental effects. In this paper, the problem of proposing maintenance optimal policy for a partially observable, stochastically deteriorating system is studied, in order to maximize the average profit of the system with consideration of sustainability aspects. The modeling of this Condition Based Sustainable Maintenance (CBSM) problem is done using mathematical methods such as Partially Observable Markov Decision Process (POMDP) and Bayesian theory. A new exact method named Accelerated Vector Pruning method and other popular estimating and exact methods are applied and compared in solving the presented CBSM model and several managerial conclusions were obtained.

Keywords

20.1001.1.10263098.2020.27.3.28.8

Main Subjects

Optimal Production and Maintenance Control

References

References:

1. Chu, C., Proth, J.M., and Wolff, P. "Predictive maintenance: The one-unit replacement model", International Journal of Production Economics, 54(3), pp. 285-295 (1998).
2. Wireman, T., Benchmarking Best Practices in Maintenance Management, Industrial Press Inc (2004).
3. Waeyenbergh, G. and Pintelon, L. "A framework for maintenance concept development", International Journal of Production Economics, 77(3), pp. 299-313 (2002).
4. Rahmati, S.H.A., Ahmadi, A., and Karimi, B. "Developing simulation based optimization mechanism for a novel stochastic reliability centered maintenance problem", Scientia Iranica, Transactions E, Industrial Engineering, 25(5), pp. 2788-2806 (2018).
5. Agustiady, T.K. and Cudney, E.A. "Total productive maintenance", Total Quality Management & Business Excellence, pp. 1-8 (2018) .
6. Nielsen, J.S. and Srensen, J.D. "Computational framework for risk-based planning of inspections, maintenance and condition monitoring using discrete Bayesian networks", Structure and Infrastructure Engineering, 14(8), pp. 1082-1094 (2018).
7. Ahmad, R. and Kamaruddin, S. "An overview of timebased and condition-based maintenance in industrial application", Computers & Industrial Engineering, 63(1), pp. 135-149 (2012).
8. Takata, S., Kirnura, F., Van Houten, F.J.A.M., et al. "Maintenance: changing role in life cycle management", CIRP Annals Manufacturing Technology, 53(2), pp. 643-655 (2004).
9. Grall, A., Berenguer, C., and Dieulle, L. "A conditionbased maintenance policy for stochastically deteriorating systems", Reliability Engineering & System Safety, 76(2), pp. 167-180 (2002a).
10. Han, Y. and Song, Y.H. "Condition monitoring techniques for electrical equipment-a literature survey", Power Delivery, IEEE Transactions on, 18(1), pp. 4- 13 (2003).
11. Moya, M.C.C. "The control of the setting up of a predictive maintenance programme using a system of indicators", Omega, 32(1), pp. 57-75 (2004).
12. Jardine, A.K., Lin, D., and Banjevic, D. "A review on machinery diagnostics and prognostics implementing condition-based maintenance", Mechanical Systems and Signal Processing, 20(7), pp. 1483-1510 (2006).
13. Gupta, A. and Lawsirirat, C. "Strategically optimum maintenance of monitoring-enabled multi-component systems using continuous-time jump deterioration models", Journal of Quality in Maintenance Engineering, 12(3), pp. 306-329 (2006).
14. Iung, B. and Levrat, E. "Advanced maintenance services for promoting sustainability", Procedia CIRP, 22, pp. 15-22 (2014).
15. Jasiulewicz-Kaczmarek, M. "Sustainability: orientation in maintenance management: case study", Eco-Production and Logistics, pp. 135-154 (2013).
16. WCED, S.W.S. "World commission on environment and development", Our Common Future (1987).
17. Michaelis, L. "The role of business in sustainable consumption", Journal of Cleaner Production, 11(8), pp. 915-921 (2003).
18. Kalantary, M., Saen, R.F., and Eshlaghy, A.T. Sustainability assessment of supply chains by inverse network dynamic data envelopment analysis", Scientia Iranica, Transactions E, Industrial Engineering, 25(6), pp. 3723-3743 (2018).
19. Fotuhi-Friuzabad, M., Safdarian, A., Moeini-Aghtaie,M., et al. "Upcoming challenges of future electric power systems: sustainability and resiliency", Scientia Iranica, 23(4), pp. 1565-1577 (2016).
20. Shahidehpour, M. and Fotuhi-Friuzabad, M. "Grid modernization for enhancing the resilience, reliability, economics, sustainability, and security of electricity grid in an uncertain environment", Scientia Iranica, Transaction D, Computer Science & Engineering, Electrical, 23(6), p. 2862 (2016).
21. Ararsa, B.B. "Green maintenance: A literature survey on the role of maintenance for sustainable manufacturing", MS Thesis, Malardalen University, School of Innovation, Design and Engineering (2012).
22. Kopac, J. "Achievements of sustainable manufacturing by machining", Journal of Achievements in Materials and Manufacturing Engineering, 34(2), pp. 180-187 (2009).
23. Ben-Daya, M., Ait-Kadi, D., Duffuaa, S.O., et al. Handbook of Maintenance Management and Engineering, 7, Springer, London (2009).
24. Sari, E., Shaharoun, A.M., Ma'aram, A., et al. "Sustainable maintenance performance measures: A pilot survey in Malaysian automotive companies", Procedia CIRP, 26, pp. 443-448 (2015).
25. Nezami, F.G. and Yildirim, M.B. "A sustainability approach for selecting maintenance strategy", International Journal of Sustainable Engineering, 6(4), pp. 332-343 (2013).
26. Mishra, R.P. and Mungi, P. "A system framework for a sustainable approach to maintenance", Sustainable Operations in India, pp. 79-91 (2018).
27. Li, S. "Optimal control of production-maintenance system with deteriorating items, emission tax and pollution R&D investment", International Journal of Production Research, 52(6), pp. 1787-1807 (2014).
28. Ben-Salem, A., Gharbi, A., and Hajji, A. "Environmental issue in an alternative production-maintenance control for unreliable manufacturing system subject to degradation", The International Journal of Advanced Manufacturing Technology, 77(1-4), pp. 383- 398 (2015).
29. Hajej, Z., Rezg, N., and Gharbi, A. "Ecological optimization for forecasting production and maintenance problem based on carbon taxthem", The International Journal of Advanced Manufacturing Technology, 88(5-8), pp. 1595-1606 (2017a).
30. Ba, K., Dellagi, S., Rezg, N., et al. "Joint optimization of preventive maintenance and spare parts inventory for an optimal production plan with consideration of CO2 emission", Reliability Engineering & System Safety, 149, pp. 172-186 (2016).
31. Hajej, Z., Rezg, N., and Gharbi, "A. Joint optimization of production and maintenance planning with an environmental impact study", The International Journal of Advanced Manufacturing Technology, 93(1-4), pp. 1269-1282 (2017b).
32. Franciosi, C., Lambiase, A., and Miranda, S. "Sustainable maintenance: A periodic preventive maintenance model with sustainable spare parts management", IFAC-PapersOnLine, 50(1), pp. 13692-13697 (2017).
33. Al-Turki, U.M., Ayar, T., Yilbas, B.S., and Sahin, A.Z., Integrated Maintenance Planning in Manufacturing Systems, Cham: Springer (2014).
34. Tlili, L., Radhoui, M., and Chelbi, A. "Conditionbased maintenance strategy for production systems generating environmental damage", Mathematical Problems in Engineering, 2015, Article ID 494162 (2015). https://doi.org/10.1155/2015/494162.
35. Chouikhi, H., Khatab, A., and Rezg, N. "A conditionbased maintenance policy for a production system under excessive environmental degradation", Journal of Intelligent Manufacturing, 25(4), pp. 727-737 (2014).
36. Ben-Salem, A., Gharbi, A., and Hajji, A. "Production and uncertain green subcontracting control for an unreliable manufacturing system facing emissions", The International Journal of Advanced Manufacturing Technology, 83(9-12), pp. 1787-1799 (2016).
37. Jiang, A., Dong, N., Tam, K.L., et al. "Development and optimization of a condition-based maintenance policy with sustainability requirements for production system", Mathematical Problems in Engineering, 2018, Article ID 4187575 (2018). https://doi.org/10.1155/2018/4187575.
38. Alaswad, S. and Xiang, Y. "A review on conditionbased maintenance optimization models for stochastically deteriorating system", Reliability Engineering & System Safety, 157, pp. 54-63 (2017).
39. Papakonstantinou, K.G. and Shinozuka, M. "Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: theory", Reliability Engineering & System Safety, 130, pp. 202-213 (2014).
40. Kumar, A. and Meenakshi, N., Marketing Management, Vikas Publishing House (2011).
41. Ahmadi-Javid, A. and Ghandali, R. "An efficient optimization procedure for designing a capacitated distribution network with price-sensitive demand", Optimization and Engineering, 15(3), pp. 801-817 (2014).
42. Pak, P.K., Kim, D.W., and Jeong, B.H. "Machine maintenance policy using partially observable Markov decision process", Journal of the KSQC, 16(2), pp. 1-9 (1988).
43. Kaelbling, L.P., Littman, M.L., and Cassandra, A.R. "Planning and acting in partially observable stochastic domains", Artificial Intelligence, 101(1), pp. 99-134 (1998).
44. Pineau, J., Gordon, G., and Thrun, S. "Point-based value iteration: An anytime algorithm for POMDPs", IJCAI, 3, pp. 1025-1032 (2003).
45. Spaan, M.T. and Vlassis, N. "Perseus: Randomized point-based value iteration for POMDPs", Journal of Artificial Intelligence Research, 24, pp. 195-220 (2005).
46. Raphael, C. and Shani, G. "The skyline algorithm for POMDP value function pruning", Annals of Mathematics and Artificial Intelligence, 65(1), pp. 61-77 (2012).
47. Karmokar, A.K., Senthuran, S., and Anpalagan, A. "POMDP-based cross-layer power adaptation techniques in cognitive radio networks", Global Communications Conference, IEEE, pp. 1380-1385 (2012).
48. Roijers, D.M., Vamplew, P., Whiteson, S., et al. "A survey of multi-objective sequential decision-making", Journal of Artificial Intelligence Research, 48, pp. 67- 113 (2013).
49. Li, D. and Jayaweera, S.K. "Machine-learning aided optimal customer decisions for an interactive smart grid", IEEE Systems Journal, 9(4), pp. 1529-1540 (2015).
50. Qian, W., Liu, Q., Zhang, Z., et al. "Policy graph pruning and optimization in Monte Carlo value iteration for continuous-state POMDPs", In Computational Intelligence (SSCI), Symposium Series on IEEE, pp. 1-8 (2016).
51. Walraven, E. and Spaan, M.T. "Accelerated Vector Pruning for Optimal POMDP Solvers", In AAAI, pp. 3672-3678 (2017).
52. Bellman, R., Dynamic Programming, Courier Corporation (2013).
53. Ahuja. D.https://www.codeproject.com/Articles/9898. /Heap-Walker (2005).
54. Agrawal, R., Realff, M.J., and Lee, J.H. "MILP based value backups in partially observed Markov decision processes (POMDPs) with very large or continuous action and observation spaces", Computers & Chemical Engineering, 56, pp. 101-113 (2013).
55. Smallwood, R.D. and Sondik, E.J. "The optimal control of partially observable Markov processes over a finite horizon", Operations Research, 21(5), pp. 1071- 1088 (1973).
56. Ozgen, S. and Demirekler, M. "A fast elimination method for pruning in POMDPs", Joint German/ Austrian Conference on Artificial Intelligence, Springer International Publishing, pp. 56-68 (2016).
57. Feng, Z. and Zilberstein, S. "Region-based incremental pruning for POMDPs", 20th Conf. on Uncertainty in Artificial Intelligence, AUAI Press, pp. 146-153 (2004).
58. Cassandra, A., Littman, M.L., and Zhang, N.L. "Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes", 13th Conf. on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers Inc., pp. 54-61 (1997).
59. White, C.C. "A survey of solution techniques for the partially observed Markov decision process", Annals of Operations Research, 32(1), pp. 215-230 (1991).
60. Littman, M.L., The Witness Algorithm: Solving Partially Observable Markov Decision Processes, Brown University, Providence, RI. (1994).