Article Sidebar

Submitted
May 25, 2024
Accepted
Jan 10, 2025
Published
Jan 10, 2025
Download
PDF
Statistic
Read Counter : 0 Download : 0

Downloads

Download data is not yet available.

Main Article Content

Abstract

Reinforcement learning (RL) approaches, particularly Q-learning, have emerged as strong tools for autonomous agent training, allowing agents to acquire optimum decision-making rules through interaction with their surroundings. This research investigates the use of Q-learning in the context of training autonomous agents for robotic soccer, a complex and dynamic arena that necessitates strategic planning, coordination, and adaptation. We studied the learning progress and performance of agents taught using Q-learning in a series of experiments carried out in a simulated soccer setting. During training, agents interacted with the soccer environment, iteratively changing their Q-values in response to observable rewards and behaviors. Despite the high-dimensional and stochastic character of the soccer domain, Q-learning helped the agents develop excellent tactics and decision-making capabilities. Notably, our study found that, on average, the agents required 64 steps to reach a stable policy with an average reward of -1.

Keywords

Q-learning Reward Reinforcement Learning occer Agents

Article Details

Lisensi Creative Commons
This work is licensed under a Lisensi Creative Commons Atribusi-BerbagiSerupa 4.0 Internasional.

How to Cite
[1]
B. Paneru, B. Paneru, R. Poudyal, and K. Poudyal, “Reinforcement Learning-Based Autonomous Soccer Agents: A Study in Multi-Agent Coordination and Strategy Development”, bit-cs, vol. 6, no. 1, pp. 10-17, Jan. 2025.

References

  1. A. Sarje, A. Chawre and S. B. Nair, "Reinforcement learning of player agents in RoboCup Soccer simulation," Fourth International Conference on Hybrid Intelligent Systems (HIS'04), Kitakyushu, Japan, 2004, pp. 480-481, doi: 10.1109/ICHIS.2004.81.
  2. Celiberto, Luiz & Reinaldo, Jr & Bianchi, Reinaldo. (2005). A REINFORCEMENT LEARNING BASED TEAM FOR THE ROBOCUP 2D SOCCER SIMULATION LEAGUE.
  3. S. Sarkar, D. P. Mukherjee and A. Chakrabarti, "Reinforcement Learning for Pass Detection and Generation of Possession Statistics in Soccer," in IEEE Transactions on Cognitive and Developmental Systems, vol. 15, no. 2, pp. 914-924, June 2023, doi: 10.1109/TCDS.2022.3194103.
  4. Hu C, Xu M, Hwang K-S. An adaptive cooperation with reinforcement learning for robot soccer games. International Journal of Advanced Robotic Systems. 2020;17(3). doi:10.1177/1729881420921324
  5. Y. Ma, Z. Cao, X. Dong, C. Zhou, and M. Tan, “A multi-robot coordinated hunting strategy with dynamic alliance,” in Proceedings of the Chinese Control and Decision Conference (CCDC '09), pp. 2338–2342, chn, June 2009.
  6. F. Michaud and M. J. Matarić, “Learning from history for behavior-based mobile robots in non-stationary conditions,” Machine Learning, vol. 31, no. 1–3, pp. 141–167, 1998.
  7. M. Asada, E. Uchibe, and K. Hosoda, “Cooperative behavior acquisition for mobile robots in dynamically changing real worlds via vision-based reinforcement learning and development,” Artificial Intelligence, vol. 110, no. 2, pp. 275–292, 1999.
  8. J. Hu and M. P. Wellman, “Nash Q-learning for general-sum stochastic games,” Journal of Machine Learning Research, vol. 4, no. 6, pp. 1039–1069, 2004.
  9. T. Fujii, Y. Arai, H. Asama, and I. Endo, “Multilayered reinforcement learning for complicated collision avoidance problems,” in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 3, pp. 2186–2191, May 1998.
  10. Y. Wang, Cooperative and intelligent control of multi-robot systems using machine learning [thesis], The University of British Columbia, 2008.
  11. M. Wiering, R. Sałustowicz, and J. Schmidhuber, “Reinforcement learning soccer teams with incomplete world models,” Autonomous Robots, vol. 7, no. 1, pp. 77–88, 1999.
  12. M. L. Littman, “Markov games as a framework for multi-agent reinforcement learning,” in Proceedings of the 11th International Conference on Machine Learning, pp. 157–163, 1994.
  13. M. J. Matarić, “Reinforcement learning in the multi-robot domain,” Autonomous Robots, vol. 4, no. 1, pp. 73–83, 1997.
  14. J. H. Kim and P. Vadakkepat, “Multi-agent systems: a survey from the robot-soccer perspective,” Intelligent Automation and Soft Computing, vol. 6, no. 1, pp. 3–18, 2000.
  15. Y. Duan, B. X. Cui, and X. H. Xu, “A multi-agent reinforcement learning approach to robot soccer,” Artificial Intelligence Review, vol. 38, no. 3, pp. 193–211, 2012.
  16. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, UK, 1998.
  17. J. Hu and M. P. Wellman, “Multiagent reinforcement learning: theoretical framework and an algorithm,” in Proceedings of the 15th International Conference on Machine Learning, pp. 242–250, 1998.
  18. C. J. C. H. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, no. 3-4, pp. 279–292, 1992.
  19. M. L. Littman, “Friend-or-foe Q-learning in general-sum games,” in Proceedings of the 18th International Conference on Machine Learning (ICML '01), pp. 322–328, 2001.

DB Error: Unknown column 'Array' in 'where clause'