Cluster Synchronization for Discrete-Time Zero-Sum Graphical Games with Unknown Constrained-Input Systems

Document Type : Research Paper

Authors

1 Electrical Engineering Department, Semnan University, Semnan, Iran.

2 Electrical Engineering Department, Amirkabir University, Tehran, Iran.

Abstract

This paper addresses the synchronization issue of agents with their respective leaders in each cluster for unknown discrete-time zero-sum graphical games with constrained input. To solve the coupled Hamilton-Jacobi-Isaacs equations under the assumption of unknown dynamics, an adaptive optimal distributed technique based on value iteration heuristic dynamic programming is proposed. An actor-critic framework is employed to approximate the value functions, control policies, and worst-case disturbance policies necessary for implementing the proposed method. Additionally, neural network identifiers are utilized to determine each agent's unknown dynamics. To prevent system instability, a constraint on control inputs is incorporated into the design method. By considering disturbances in the dynamics, the proposed solutions are made robust against unpredictable events, enhancing performance and stability. Furthermore, the closed-loop system's stability is proven. Finally, the theoretical results are validated through simulation outcomes.

Keywords

Main Subjects

References

[1] T. Liu and Z. P. Jiang, “Distributed control of multi-agent systems with pulse-width-modulated controllers,” Automatica, vol. 119, pp. 109020–109020, Sep. 2020‌.
[2] X. Ge, Q. L. Han, L. Ding, Y. L Wang, X. M. Zhang, “Dynamic event-triggered distributed coordination control and its applications: A survey of trends and techniques,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 50, no. 9, pp. 3112-3125, 2020.
[3] P. Zhang, H. Xue, S. Gao, J. Zhang, “Distributed adaptive consensus tracking control for multi-agent system with communication constraints,” IEEE Transactions on Parallel and Distributed Systems vol. 32, no. 6 pp. 1293-1306, 2020.
[4] J. Huang, W. Wang, C. Wen, J. Zhou, G. Li, “Distributed adaptive leader-follower and leaderless consensus control of a class of strict-feedback nonlinear systems: A unified approach,” Automatica, vol. 118, pp. 109021, 2020.
[5] S. Zhai, X. Wang, W. X. Zheng, "Leaderless cluster consensus of second-order general nonlinear multiagent systems under directed topology." IEEE Transactions on Circuits and Systems I: Regular Papers, 2023.
[6] K. Chen, J. Wang, X. Zeng, Y. Zhang, F.L. Lewis, “Cluster output regulation of heterogeneous multi-agent systems,” International Journal of Control, vol. 93, no. 12, pp. 2973-2981, 2020.
[7] J. Qin, W. Fu, Y. Shi, H. Gao, Y. Kang, “Leader-following practical cluster synchronization for networks of generic linear systems: An event-based approach,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 1, pp. 215-224, 2018.
[8] Y. Wang, Z. Ma, J. Cao, A. Alsaedi, F. E. Alsaadi, “Adaptive cluster synchronization in directed networks with nonidentical nonlinear dynamics,” Complexity, vol. 21, no. s2, pp. 380–387, 2016.
[9] Y. Han, W. Lu, T. Chen, “Cluster consensus in discrete-time networks of multi-agents with inter-cluster nonidentical inputs,” IEEE Transactions on Neural Networks and Learning Systems vol. 24, no. 4 pp. 566-578, 2023.
[10] Qin, Jiahu, and Changbin Yu. “Cluster consensus control of generic linear multi-agent systems under directed topology with acyclic partition,” Automatica, vol. 49, no. 9, pp. 2898-2905, 2013.
[11] T. Basar and Geert Jan Olsder, “Dynamic noncooperative game theory. 2nd ed. (Classics in applied mathematics 23),” Jan. 1999.
[12] K. G. Vamvoudakis, F. L. Lewis, and G. Hudas, “Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimality,” Automatica, vol. 48, no. 8, pp. 1598–1611, Aug. 2012.
[13] Q. Wei, D. Liu, and F. L. Lewis, “Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games,” Information Sciences, vol. 317, pp. 96–113, Oct. 2015.
‌[14] F. Tatari, M.B. Naghibi-Sistani, and K. G. Vamvoudakis, “Distributed optimal synchronization control of linear networked systems under unknown dynamics,” American Control Conference(ACC), May 2017.
[15] F. Tatari, M.B. Naghibi-Sistani, and K. G. Vamvoudakis, “Distributed learning algorithm for non-linear differential graphical games,” Transactions of the Institute of Measurement and Control, vol. 39, no. 2, pp. 173–182, Jul. 2016.
‌[16] Q. Jiao, H. Modares, S. Xu, F.L. Lewis, and K.G. Vamvoudakis, “Multi-agent zero-sum differential graphical games for disturbance rejection in distributed control,” Automatica, vol. 69, pp. 24-34, 2016.
[17] F. Tatari, K.G. Vamvoudakis, and M.  Mazouchi, “Optimal distributed learning for disturbance rejection in networked nonlinear games under unknown dynamics,” IET Control Theory and Applications, vol. 13, no. 17, pp.2838-2848, 2019.
[18] D. Liu, H. Li, and D. Wang, “Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm,” Neurocomputing, vol. 110, pp. 92–100, 2013.
‌[19] B. Kiumarsi, H. Modares, F.L. Lewis, and Z.P.  Jiang, “H-infinity optimal control of unknown linear discrete-time systems: An off-policy reinforcement learning approach,” 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), pp. 41-46, 2015.
[20] R. Song and L. Zhu, “Stable value iteration for two-player zero-sum game of discrete-time nonlinear systems based on adaptive dynamic programming,” Neurocomputing, vol. 340, pp. 180–195, May 2019.
‌[21] M. Abouheaf, F. L. Lewis, K. G. Vamvoudakis, Sofie Haesaert, and R. Babuska, “Multi-agent discrete-time graphical games and reinforcement learning solutions,” Automatica, vol. 50, no. 12, pp. 3038–3053, 2014.
‌[22] M. I. Abouheaf, F. L. Lewis, M. S. Mahmoud, and D. G. Mikulski, “Discrete-time dynamic graphical games: model-free reinforcement learning solution,” Control Theory and Technology, vol. 13, no. 1, pp. 55–69, Feb. 2015.
[23] R. S. Sutton and A. G. Barto, “Reinforcement Learning: An Introduction,” Robotica, vol. 17, no. 2, pp. 229-235, 1999.
[24] B. Kiumarsi, H. Modares, and F. Lewis, “Reinforcement Learning for Distributed Control and Multi-player Games,” In Handbook of Reinforcement Learning and Control, Springer, Cham, pp. 7-27, 2021.
[25] H. Jiang, H. Zhang, G. Xiao, and X. Cui, “Data-based approximate optimal control for nonzero-sum games of multi-player systems using adaptive dynamic programming” Neurocomputing, 275, pp. 192-199, 2018.
[26] P. Liu, H. Zhang, C. Liu, and H. Su, “Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints,” IEEE Access, vol. 8, pp. 182295–182306, Jan. 2020.
[27] B. Luo, D. Liu, and H.-N. Wu, “Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2099–2111, 2018.
[28] Yang, Ni, Jiang-Wen Xiao, Li Xiao, and Yan-Wu Wang. “Non-zero sum differential graphical game: cluster synchronization for multi-agents with partially unknown dynamics,” International Journal of Control. vol, 92, no. 10 pp. 2408-2419, 2019.
[29] Z. Jahan, A. Dideban, M. Arab Khabori, “ Cluster synchronization for unknown constrained-input discrete-time graphical games with reinforcement learning algorithms”, The first artificial intelligence and intelligent processing conference, Iran, Semnan, 2022.
[30] S. Khoo, L. Xie, and Z. Man, “Robust Finite-Time Consensus Tracking Algorithm for Multirobot Systems,” IEEE/ASME Transactions on mechatronics, vol. 14, no. 2, pp. 219–228, Mar. 2009.
[31] H. Zhang, Y. Luo, & D. Liu, “Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints” IEEE Transactions on Neural Networks, vol. 20, no. 9, pp. 1490–1503, 2009.

Files

History

  • Receive Date: 20 April 2024
  • Revise Date: 21 October 2024
  • Accept Date: 17 February 2025

Share

How to cite

Statistics