Simulation-based optimization of a stochastic supply chain considering supplier disruption: Agent-based modeling and reinforcement learning

Document Type : Article

Authors

Department of Industrial Engineering, K.N. Toosi University of Technology, Pardis Street, Mollasadra Street, Vanaq Square, Tehran, 1999143344, Iran

Abstract

Many researchers and practitioners in the recent years have been attracted to investigate the role of uncertainties in the supply chain management concept. In this paper a multi-period stochastic supply chain with demand uncertainty and supplier disruption is modeled. In the model, two types of retailers including risk sensitive and risk neutral, with many capacitated suppliers are considered. Autonomous retailers have three choices to satisfy demands: ordering from primary suppliers, reserved suppliers and spot market. The goal is to find the best behavior of the risk sensitive retailer, regarding the forward and option contracts, during several contract periods based on the profit function. Hence, an agent-based simulation approach has been developed to simulate the supply chain and transactions between retailers and unreliable suppliers. In addition, a Q-learning approach (as a method of reinforcement learning) has been developed to optimize the simulation procedure. Furthermore, different configurations for simulation procedure are analyzed. The R-netlogo package is used to implement the algorithm. Also a numerical example has been solved using the proposed simulation-optimization approach. Several sensitivity analyzes are conducted regarding different parameters of the model. Comparison of the numerical results with a genetic algorithm shows a significant efficiency of the proposed Q-leaning approach.

Keywords

Main Subjects

References

References:
1. Merzifonluoglu, Y. "Risk averse supply portfolio selection with supply, demand and spot market volatility", Omega, 57, pp. 40-53 (2015).
2. Ray, P. and Jenamani, M. "sourcing under supply disruption with capacity-constrained suppliers", Journal of Advances in Management Research, 10(2), pp. 192- 205 (2013).
3. Ray, P. and Jenamani, M. "sourcing decision under disruption risk with supply and demand uncertainty: A newsvendor approach", Annals of Operations Research, 237(1), pp. 237-262 (2016).
4. Kim, G., Wu, K., and Huang, E. "Optimal inventory control in a multi-period newsvendor problem with non-stationary demand", Advanced Engineering Informatics, 29(1), pp. 139-145 (2015).
5. Chopra, S. and Meidl, P., Supply Chain Management: Strategy, Planning and Operation, Pearson, Sixth edition, USA (2016).
6. Chiacchio, F., Pennisi, M., Russo, G., Motta, S., and Pappalardo, F. "Agent-based modeling of the immune system: NetLogo, a promising framework", BioMed Research International, 2, pp. 1-6 (2014).
7. Humann, J. and Madni, A.M. "Integrated agentbased modeling and optimization in complex systems analysis", Procedia Computer Science, 28, pp. 818-827 (2014).
8. Macal, C.M. "Everything you need to know about agent-based modelling and simulation", Journal of Simulation, 10, pp. 144-156 (2016).
9. Avci, M.G. and Selim, H. "A multi-objective, simulation-based optimization framework for supply chains with premium freights", Expert Systems with Applications, 67, pp. 95-106 (2017).
10. Sutton, R.S. and Barto, A.G., Reinforcement Learning: An Introduction, MIT press, Cambridge (1998).
11. Gosavi, A. "Reinforcement learning for long-run average cost", European Journal of Operational Research, 155, pp. 654-674 (2004).
12. Merzifonluoglu, Y. and Feng, Y. "Newsvendor problem with multiple unreliable suppliers", International Journal of Production Research, 52(1), pp. 221-242 (2014).
13. Merzifonluoglu, Y. "Impact of risk aversion and backup supplier on sourcing decisions of a firm", International Journal of Production Research, 53(22), pp. 6937-6961 (2015).
14. Merzifonluoglu, Y. "Integrated demand and procurement portfolio management with spot market volatility and option contracts", European Journal of Operational Research, 258(1), pp. 181-192 (2017).
15. Bouakiz, M. and Sobel, M.J. "Inventory control with an exponential utility criterion", Operations Research, 40(3), pp. 603-608 (1992).
16. Wang, H.F., Chen, B.C., and Yan, H.M. "Optimal inventory decisions in a multi period newsvendor problem with partially observed Markovian supply capacities", European Journal of Operational Research, 202, pp. 502-517 (2010).
17. Densing, M. "Dispatch planning using newsvendor dual problems and occupation times: application to hydropower", European Journal of Operational Research, 228, pp. 321-330 (2013).
18. Jalali, H. and Nieuwenhuyse, I.V. "Simulation optimization in inventory replenishment: a classification", IIE Transactions, 47, pp. 1217-1235 (2015).
19. Nikolopoulou, A. and Ierapetritou, M.G. "Hybrid simulation based optimization approach for supply chain management", Computers & Chemical Engineering, 47, pp. 183-193 (2012).
20. Kwon, O., Im, G.P., and Lee, K.C. "MACE-SCM: A multi-agent and case-based reasoning collaboration mechanism for supply chain management under supply and demand uncertainties", Expert Systems with Applications, 33(3), pp. 690-705 (2007).
21. Chaharsooghi, S.K., Heydari, J., and Zegordi, S.H. "A reinforcement learning model for supply chain ordering management: An application to the beer game", Decision Support Systems, 45(4), pp. 949-959 (2008).
22. Sun, R. and Zhao, G. "Analyses about efficiency of reinforcement learning to supply chain ordering management", IEEE 10th International Conference on Industrial Informatics, China (2012).
23. Dogan, I. and Guner, A.R. "A reinforcement learning approach to competitive ordering and pricing problem", Expert Systems, 32(1), pp. 39-48 (2015).
24. Jiang, C. and Sheng, Z. "Case-based reinforcement learning for dynamic inventory control in a multi-agent supply-chain system", Expert Systems with Applications, 36(3), pp. 6520-6526 (2009).
25. Kim, C.O., Kwon, I.-H., and Kwak, C. "Multiagent based distributed inventory control model", Expert Systems with Applications, 37(7), pp. 5186-5191 (2010).
26. Mortazavi, A., Khamseh, A.A., and Azimi, P. "Designing of an intelligent self-adaptive model for supply chain ordering management system", Engineering Applications of Artificial Intelligence, 37, pp. 207-220 (2015).
27. Rabe, M. and Dross, F. "A reinforcement learning approach for a decision support system for logistics networks", Winter Simulation Conference, USA (2015).
28. Zhou, J., Purvis, M., and Muhammad, Y. "A combined modelling approach for multi-agent collaborative planning in global supply chains", 8th International Symposium on Computational Intelligence and Design, China (2015).
29. Thiele, J. and Marries, R. "NetLogo: introduction to the RNetLogo package", Journal of Statistical Software, 58, pp. 1-41 (2014).
30. Liu, R., Tao, Y., Hu, Q., and Xie, X. "Simulationbased optimisation approach for the stochastic twoechelon logistics problem", International Journal of Production Research, 55(1), pp. 187-201 (2017).
Scientia Iranica
Volume 26, Issue 6 - Serial Number 6
Transactions on Industrial Engineering (E)
November and December 2019
Pages 3780-3795

Files

History

  • Receive Date: 20 June 2017
  • Revise Date: 06 April 2018
  • Accept Date: 21 July 2018

Share

How to cite

Statistics