Multi-Agent Reinforcement Learning:
Foundations and Modern Approaches

Stefano V. Albrecht, Filippos Christianos, Lukas Schäfer

Published by MIT Press, 2024

The first comprehensive introduction to the field of multi-agent reinforcement learning, an area of machine learning in which multiple decision-making agents learn to optimally interact in a shared environment. The book can be ordered from stores (see MIT Press page), and the PDF version, algorithm code, and slides are freely available from this page.

Download

Please report errata via e-mail to: issues~at~marl-book~dot~com

Download PDF

News

Follow the authors on X to get updates: Stefano, Filippos, Lukas

The Artificial Intelligence Research Institute in Barcelona (IIIA-CSIC) hosted a summer course based on the book given by Stefano V. Albrecht. The lecture recordings are now available.
The book can now be pre-ordered (e.g. MIT Press, Amazon, Penguin Random House) with scheduled release on 17 December 2024.
A Chinese translated edition of the book will be published by China Machine Press Co. in December 2025.

Citation

Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024.

@book{ marl-book,
  author = {Stefano V. Albrecht and Filippos Christianos and Lukas Sch\"afer},
  title = {Multi-Agent Reinforcement Learning: Foundations and Modern Approaches},
  publisher = {MIT Press},
  year = {2024},
  url = {https://www.marl-book.com}
}

Codebase

The book comes with a codebase written in the Python programming language, which contains implementations of several MARL algorithms presented in the book. The primary purpose of the codebase is to provide algorithm code that is self-contained and easy to read.

The codebase can be accessed via the book's GitHub repository.

Slides

Lecture slides for chapters 1 to 9 are available. We provide both PDF files and the latex source files so that instructors can make changes as required. If you use the book or slides in your class, please let us know and we will add your class and institution to a class list.

The slides (PDF and latex source files) can be accessed via the book's GitHub repository.

Summary of Notation
List of Figures
Preface

Chapter 1: Introduction

Part 1: Foundations of Multi-Agent Reinforcement Learning

Chapter 2: Reinforcement Learning
Chapter 3: Games: Models of Multi-Agent Interaction
Chapter 4: Solution Concepts for Games
Chapter 5: Multi-Agent Reinforcement Learning in Games: First Steps and Challenges
Chapter 6: Multi-Agent Reinforcement Learning: Foundational Algorithms

Part 2: Multi-Agent Deep Reinforcement Learning: Algorithms and Practice

Chapter 7: Deep Learning
Chapter 8: Deep Reinforcement Learning
Chapter 9: Multi-Agent Deep Reinforcement Learning
Chapter 10: Multi-Agent Deep Reinforcement Learning in Practice
Chapter 11: Multi-Agent Environments

Appendix A: Surveys on Multi-Agent Reinforcement Learning

Endorsements (solicited by MIT Press)

"The authors meticulously bring reinforcement learning together with game theory to provide a foundation for research and application of multi-agent reinforcement learning. This book is the perfect starting point for a grounding in the field."
—Andrew Barto, Professor Emeritus, University of Massachusetts Amherst

"Multi-agent reinforcement learning is well positioned to be the next white-hot area of artificial intelligence and this book provides the essential background, concepts, and insights for understanding this exciting and important area of research."
—Michael L. Littman, Professor of Computer Science, Brown University

"This book is the first complete reference for the growing area of multi-agent reinforcement learning. It provides both an essential resource for newcomers to the field and a valuable perspective for established researchers."
—Peter Stone, Professor of Computer Science, The University of Texas at Austin

"A landmark textbook to multiagent reinforcement learning, combining game-theoretic foundations with state-of-the-art deep learning. This essential textbook delivers fundamental insights for newcomers, experts and practitioners, featuring real-world applications and advanced algorithms."
—Karl Tuyls, Professor of Computer Science, University of Liverpool

"This will become the standard text of the emerging field of multiagent reinforcement learning. It builds from foundational ideas, incorporating recent breakthroughs in deep learning. This book will help accelerate theoretical and practical progress."
—Mykel J. Kochenderfer, Professor of Aeronautics and Astronautics, Stanford University

Multi-Agent Reinforcement Learning: Foundations and Modern Approaches