Multi-Agent Reinforcement Learning:
Foundations and Modern Approaches
Stefano V. Albrecht, Filippos Christianos, Lukas Schäfer
Published by MIT Press, 2024
The first comprehensive introduction to the field of multi-agent reinforcement learning, an area of machine learning in which multiple decision-making agents learn to optimally interact in a shared environment. The book can be ordered from stores (see MIT Press page), and the PDF version, algorithm code, and slides are freely available from this page.
Download
Please report errata via e-mail to: issues~at~marl-book~dot~com
News
Follow the authors on X to get updates: Stefano, Filippos, Lukas
- The Artificial Intelligence Research Institute in Barcelona (IIIA-CSIC) hosted a summer course based on the book given by Stefano V. Albrecht. The lecture recordings are now available on YouTube.
- The book can now be pre-ordered (e.g. MIT Press, Amazon, Penguin Random House) with scheduled release on 17 December 2024.
- A Chinese translated edition of the book will be published by China Machine Press Co. in December 2025.
Citation
Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024.
@book{ marl-book,
author = {Stefano V. Albrecht and Filippos Christianos and Lukas Sch\"afer},
title = {Multi-Agent Reinforcement Learning: Foundations and Modern Approaches},
publisher = {MIT Press},
year = {2024},
url = {https://www.marl-book.com}
}
Codebase
The book comes with a codebase written in the Python programming language, which contains implementations of several MARL algorithms presented in the book. The primary purpose of the codebase is to provide algorithm code that is self-contained and easy to read.
The codebase can be accessed via the book's GitHub repository.
Slides
Lecture slides for chapters 1 to 9 are available. We provide both PDF files and the latex source files so that instructors can make changes as required. If you use the book or slides in your class, please let us know and we will add your class and institution to a class list.
The slides (PDF and latex source files) can be accessed via the book's GitHub repository.
Table of contents
Summary of Notation
List of Figures
Preface
- Chapter 1: Introduction
Part 1: Foundations of Multi-Agent Reinforcement Learning
- Chapter 2: Reinforcement Learning
- Chapter 3: Games: Models of Multi-Agent Interaction
- Chapter 4: Solution Concepts for Games
- Chapter 5: Multi-Agent Reinforcement Learning in Games: First Steps and Challenges
- Chapter 6: Multi-Agent Reinforcement Learning: Foundational Algorithms
Part 2: Multi-Agent Deep Reinforcement Learning: Algorithms and Practice
- Chapter 7: Deep Learning
- Chapter 8: Deep Reinforcement Learning
- Chapter 9: Multi-Agent Deep Reinforcement Learning
- Chapter 10: Multi-Agent Deep Reinforcement Learning in Practice
- Chapter 11: Multi-Agent Environments
Appendix A: Surveys on Multi-Agent Reinforcement Learning
Endorsements (solicited by MIT Press)
"The authors meticulously bring reinforcement learning together with game theory to provide a foundation for research and application of multi-agent reinforcement learning. This book is the perfect starting point for a grounding in the field."
—Andrew Barto, Professor Emeritus, University of Massachusetts Amherst
"Multi-agent reinforcement learning is well positioned to be the next white-hot area of artificial intelligence and this book provides the essential background, concepts, and insights for understanding this exciting and important area of research."
—Michael L. Littman, Professor of Computer Science, Brown University
"This book is the first complete reference for the growing area of multi-agent reinforcement learning. It provides both an essential resource for newcomers to the field and a valuable perspective for established researchers."
—Peter Stone, Professor of Computer Science, The University of Texas at Austin
"A landmark textbook to multiagent reinforcement learning, combining game-theoretic foundations with state-of-the-art deep learning. This essential textbook delivers fundamental insights for newcomers, experts and practitioners, featuring real-world applications and advanced algorithms."
—Karl Tuyls, Professor of Computer Science, University of Liverpool
"This will become the standard text of the emerging field of multiagent reinforcement learning. It builds from foundational ideas, incorporating recent breakthroughs in deep learning. This book will help accelerate theoretical and practical progress."
—Mykel J. Kochenderfer, Professor of Aeronautics and Astronautics, Stanford University