- Title
- A comparison study of cooperative Q-learning algorithms for independent learners
- Creator
- Abed-Alguni, Bilal H.; Paul, David J.; Chalup, Stephan K.; Henskens, Frans A.
- Relation
- International Journal of Artificial Intelligence Vol. 14, Issue 1, p. 71-93
- Relation
- http://www.ceser.in/ceserp/index.php/ijai/article/view/4253
- Publisher
- Centre for Environment, Social and Economic Research Publications
- Resource Type
- journal article
- Date
- 2016
- Description
- Cooperative reinforcement learning algorithms such as BEST-Q, AVE-Q, PSO-Q, and WSS use Q-value sharing strategies between reinforcement learners to accelerate the learning process. This paper presents a comparison study of the performance of these cooperative algorithms as well as an algorithm that aggregates their results. In addition, this paper studies the effects of the frequency of Q-value sharing on the learning speed of the independent learners that share their Q-values among each other. The algorithms are compared using the taxi problem (multi-task problem) and different instances of the shortest path problem (single-task problem). The experimental results when learners have equal levels of experience suggest that sharing of Q-values is not beneficial and produces similar results to single agent Q-learning. However, the experimental results when learners have different levels of experience suggest that most of the cooperative Q-learning algorithms perform similarly, but better than single agent Q-learning, especially when Q-value sharing is highly frequent. This paper then places Q-value sharing in the context of modern reinforcement learning techniques and suggests some future directions for research.
- Subject
- cooperative learning; Q-learning; Q-values; Q-value sharing strategy; single-agent system; multi-agent system
- Identifier
- http://hdl.handle.net/1959.13/1331077
- Identifier
- uon:26532
- Identifier
- ISSN:0974-0635
- Language
- eng
- Full Text
- Reviewed
- Hits: 14852
- Visitors: 9710
- Downloads: 1227
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | ATTACHMENT02 | Publisher version (open access) | 304 KB | Adobe Acrobat PDF | View Details Download |