Introduction
On September 20th and 21st 2023, the School for Information and Knowledge Systems (SIKS) will be organising a new two-day course on Reinforcement Learning for Adaptive Hybrid Intelligence.
Reinforcement learning (RL) is one of the main paradigms of machine learning, in which artificial agents learn optimal behaviour from interaction data. Recent years have seen notable breakthroughs in this field in robotics, games such as Atari and Go, and in ChatGPT. While the focus on autonomous and active learning makes reinforcement learning a powerful tool, deployment of a reinforcement learning agent as an assistant or collaborator (that is, as a hybrid intelligence) raises specific challenges.
In this two-day course we will take a look at the foundations of reinforcement learning and various extensions that are important for hybrid intelligence. These include:
- “Learning under (symbolic) constraints”: safety, ethical, or legal constraints can shape the reward function and eventually help the artificial agent in exploration vs. exploitation trade-off
- “Causal reinforcement learning” will take on designing RL agents that can interact with the environment in both explaining the data generation process in the environment, and making better decisions provided that they are equipped with causal knowledge
- “Learning human preferences” will focus on obtaining reward functions from data or interactions where these are hard to specify manually
- “Multi-agent RL (MARL)” will focus on the setting where multiple RL agents interact in a shared environment, leading to game-theoretic dynamics
- “Multi-objective RL” will focus on scenarios where the agent should take into account multiple, potentially conflicting, criteria and goals.
Although the course is primarily intended for SIKS Ph.D. students, other participants will not be excluded. However, their number of passes will be restricted and will depend on the number of SIKS Ph.D. students participating in the course.
Course coordinators
Rineke Verbrugge (RUG), Erman Acar (UvA) and Herke van Hoof (UvA)
Location
The course will take place at hotel Mitland near Utrecht city centre.
Programme
Wednesday 20 September 2023: Reinforcement Learning
9:15-9:30 Doors open
9:30-9:45 Course directors: Welcome and Introduction
9:45-10:45 Reinforcement Learning ‘Advanced Basics’ I – Herke van Hoof
10:45-11:15 Coffee break
11:15-12:15 Reinforcement Learning ‘Advanced Basics’ II – Herke van Hoof
12:15-13:30 LUNCH
13:30-14:15 Safe reinforcement learning – Matthijs Spaan
14:15-15:15 Learning policies that generalize in a reliable and systematic way – Hector Geffner
15:15-15:30 Coffee break
15:30-16:15 Causality and reinforcement learning – Erman Acar
16:15-17:00 Multi-objective learning & preference elicitation – Diederik Roijers
17:30 Dinner
Thursday 21 September 2023: Multi-Agent Reinforcement Learning
9:30-10:30 Basics of multi-agent reinforcement learning, including game theory – Ann Nowé
10:30 – 11:00 Poster session I
11:00-11:30 Coffee break
11:30-12:15 Communication in multi-agent Reinforcement Learning – Shihan Wang
12:15-13:30 LUNCH
13:30-14:30 Cooperation fallacies: zero-shot coordination and relative overgeneralization –Wendelin Boehmer
14:30-15:00 Poster session II
15:00-15:30 Coffee break
15:30-16:15 RL for ad-hoc collaboration – Mustafa Mert Çelikok
16:15-17:00 Inverse reinforcement learning and imitation learning in robotics – Jens Kober
17:00-17:05 Closing – course directors
Registration
Registration for the course is now closed
