Journal of Optimization and Supply Chain Management

Journal of Optimization and Supply Chain Management

Learning to Bargain: A Game-Theoretic Reinforcement Learning Framework for Dynamic Profit-Sharing in Manufacturer–Retailer Supply Chains with Strategi

Document Type : SI: ORML 2025

Authors
Serap Ergun 1
Sirma Zeynep Alparslan Gok 2
1 Department of Computer Engineering, Faculty of Technology, Isparta University of Applied Sciences, Isparta, Türkiye
2 Department of Mathematics, Faculty of Engineering and Natural Sciences, Süleyman Demirel University, 32260, Isparta, Türkiye
10.22034/ISS.2026.10044.1066
Abstract
Objective: Traditional supply chain coordination mechanisms have limited adaptability under demand uncertainty and strategic agent behavior. This study proposes a game-theoretic multi-agent reinforcement learning (MARL) framework for dynamic profit-sharing and decentralized negotiation in manufacturer–retailer supply chains.
Methods: The framework integrates Rubinstein-style bargaining with Proximal Policy Optimization (PPO)-based MARL, enabling self-interested agents to negotiate wholesale prices, order quantities, and revenue-sharing terms over repeated interactions. Strategic customer behavior, including price sensitivity and stockout-driven demand shifts, is modeled using a real-world Walmart retail dataset. The framework is compared with centralized optimization, fixed revenue-sharing contracts, and myopic Nash bargaining policies. Robustness, sensitivity, and SHAP-based explainability analyses are also performed. 
Results: The proposed MARL framework increases total supply chain profit by 12.8% and reduces stockout rates by 68.3% compared with fixed-contract coordination. The learned policies remain stable under varying demand and customer behavior scenarios. SHAP analysis identifies inventory levels and prior-period stockouts as the main drivers of negotiation decisions.
Conclusion: The results demonstrate that integrating bargaining theory with explainable MARL provides an adaptive alternative to static supply chain coordination. The framework supports decentralized negotiation, improves economic and service outcomes, and enhances the managerial interpretability of learned policies
Keywords
supply chain coordination
game theory
reinforcement learning
multi-agent systems
bargaining
Subjects
Aviv, Y., & Wei, M. M. (2021). Strategic customer behavior, commitment, and supply chain performance. Management Science, 67(8), 4901–492. https://doi.org/10.1287/mnsc.2020.3681
Bozkir, A., Shi, Y., & Wu, J. (2023). Predicting retail sales with machine learning: A comprehensive evaluation. Expert Systems with Applications, 218, 119572.
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016). OpenAI Gym. arXiv. https://arxiv.org/abs/1606.01540
Cachon, G. P. (2003). Supply chain coordination with contracts. In S. C. Graves & A. G. de Kok (Eds.), Handbooks in operations research and management science: Vol. 11. Supply chain management: Design, coordination and operation (pp. 227–339). Elsevier.
Cachon, G. P., & Lariviere, M. A. (2005). Supply chain coordination with revenue-sharing contracts: Strengths and limitations. Management Science, 51(1), 30–44.
Cachon, G. P., & Swinney, R. (2009). Purchasing, pricing, and quick response in the presence of strategic consumers. Management Science, 55(3), 497–511.
Chen, L., Misra, S., & Moallemi, C. C. (2023). Dynamic pricing with strategic customers: A reinforcement learning approach. Manufacturing & Service Operations Management, 25(2), 542–560.
Chen, X., Federgruen, A., & Heching, A. (2015). A dynamic inventory model with strategic customer behavior. Management Science, 61(5), 1071–1089.
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785
Ergün, S., Usta, P., Alparslan Gök, S. Z., & Weber, G. W. (2023). A game theoretical approach to emergency logistics planning in natural disasters. Annals of Operations Research, 324(1), 855–868.
Ferreira, K. J., Lee, B. H., & Simchi-Levi, D. (2016). Analytics for an online retailer: Demand forecasting and price optimization. Manufacturing & Service Operations Management, 18(1), 69–88.
Gijsbrechts, J., Boute, R. N., Van Mieghem, J. A., & Zhang, D. (2022). Can deep reinforcement learning improve inventory management? Performance on dual sourcing, lost sales, and multi-echelon problems. Manufacturing & Service Operations Management, 24(3), 1249–1268.
Alparslan Gök, S.Z., Ergün, S., Kırlar, B. B., Özcan, İ., & Tayman, A. (2023). Fuzzy perspective of online games by using cryptography and cooperative game theory. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 31(6), 891–915.
Heide, J. B., & John, G. (1992). Do norms matter in marketing relationships? Journal of Marketing, 56(2), 69–85.
Kesavan, S., & Kushwaha, T. (2020). Estimating lost sales in retail operations. Production and Operations Management, 29(7), 1680–1698.
Kırlar, B. B., Ergün, S., Alparslan Gök, S. Z., & Weber, G. W. (2018). A game-theoretical and cryptographical approach to crypto-cloud computing and its economical and financial aspects. Annals of Operations Research, 260(1), 217–231.
Lewis, M., Yarats, D., Dauphin, Y. N., Parikh, D., & Batra, D. (2017). Deal or no deal? End-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (pp. 2443–2453).
Li, J., Chen, X., Ni, L. M., & Tang, Y. (2021). Multi-agent reinforcement learning for urban logistics: A survey. Transportation Research Part C: Emerging Technologies, 132, Article 103393.
Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
Mao, H., et al. (2020). Robotic scheduling in automated warehouses via reinforcement learning.
Muthoo, A. (1999). Bargaining theory with applications. Cambridge University Press.
Nagarajan, M., & Sošić, G. (2008). Game-theoretic analysis of cooperation among supply chain agents: Review and extensions. European Journal of Operational Research, 187(3), 719–745.
Nash, J. F. (1950). The bargaining problem. Econometrica, 18(2), 155–162.
SteadieSeifi, M., Dellaert, N. P., Nuijten, W., Van Woensel, T., & Raoufi, R. (2014). Multimodal freight transportation planning: A literature review. European Journal of Operational Research, 233(1), 1–15.
Oroojlooyjadid, A., Nazari, M., Snyder, L. V., & Takac, M. (2020). A deep Q-network for inventory control in a lost-sales setting. European Journal of Operational Research, 285(1), 159–170.
Rubinstein, A. (1982). Perfect equilibrium in a bargaining model. Econometrica, 50(1), 97–109.
Su, X. (2007). Intertemporal pricing with strategic customer behavior. Management Science, 53(5), 726–741.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
Tan, M. (1993). Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceedings of the Tenth International Conference on Machine Learning (pp. 330–337). Morgan Kaufmann.
Terry, J., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R., Santos, L. S., Dieffendahl, C., Horsch, C., Perez-Vicente, R., Williams, N., Lokesh, Y., & Ravi, P. (2021). PettingZoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34, 15032–15043.
Tsay, A. A., & Agrawal, N. (2000). Channel dynamics under price and service competition. Manufacturing & Service Operations Management, 2(4), 372–391. https://doi.org/10.1287/msom.2.4.372.12351
Walmart Recruiting Challenge Dataset. (2023). Kaggle.
Xu, H., Dang, C., & He, N. (2023). Robust reinforcement learning with Wasserstein distributional robustness. Operations Research, 71(4), 1274–1292.
Zhang, Y., Wang, J., & Zhang, D. (2021). Deep reinforcement learning for automated negotiation in e-commerce. Electronic Commerce Research and Applications, 48, Article 101063. https://doi.org/10.1016/j.elerap.2021.101063
Zhang, W., Kong, M., Zhang, Y., Fathollahi-Fard, A. M., & Tian, G. (2025). A revised deep reinforcement learning algorithm for parallel machine scheduling problem under multi-scenario due date constraints. Swarm and Evolutionary Computation, 92, 101808.
Zhang, W., Zuo, C., Song, Y., Kong, M., Fathollahi-Fard, A. M., & Gao, K. (2026). Reinforcement learning-enhanced metaheuristic algorithm for a battery recycling closed-loop supply chain network design problem. Research in Transportation Business & Management, 68, 101768.
Zhang, W., Kong, M., Tan, W., Song, Y., & Fathollahi-Fard, A. M. (2025). A reinforcement learning-driven framework for the Q-commerce multi-product unit scheduling problem. Electronic Commerce Research and Applications, 101571.
Zhang, W., Zuo, C., Kong, M., Zhang, T., Fathollahi-Fard, A. M., & Dulebenets, M. A. (2026). Human-robot collaborative supply chain scheduling via reinforcement learning, large neighborhood search, and large language models. Journal of Industrial Information Integration, 101152.
Journal of Optimization and Supply Chain Management
Volume 3, Issue 3
Summer 2026
Pages 261-291