SIKS course on Reinforcement Learning for Adaptive Hybrid Intelligence


On September 20th and 21st 2023, the School for Information and Knowledge Systems (SIKS) will be organising a new two-day course on Reinforcement Learning for Adaptive Hybrid Intelligence.

Reinforcement learning (RL) is one of the main paradigms of machine learning, in which artificial agents learn optimal behaviour from interaction data. Recent years have seen notable breakthroughs in this field in robotics, games such as Atari and Go, and in ChatGPT. While the focus on autonomous and active learning makes reinforcement learning a powerful tool, deployment of a reinforcement learning agent as an assistant or collaborator (that is, as a hybrid intelligence) raises specific challenges.

In this two-day course we will take a look at the foundations of reinforcement learning and various extensions that are important for hybrid intelligence. These include:

  • “Learning under (symbolic) constraints”: safety, ethical, or legal constraints can shape the reward function and eventually help the artificial agent in exploration vs. exploitation trade-off
  • “Causal reinforcement learning” will take on designing RL agents that can interact with the environment in both explaining the data generation process in the environment, and making better decisions provided that they are equipped with causal knowledge
  • “Learning human preferences” will focus on obtaining reward functions from data or interactions where these are hard to specify manually
  • “Multi-agent RL (MARL)” will focus on the setting where multiple RL agents interact in a shared environment, leading to game-theoretic dynamics
  • “Multi-objective RL” will focus on scenarios where the agent should take into account multiple, potentially conflicting, criteria and goals. 

Although the course is primarily intended for SIKS Ph.D. students, other participants will not be excluded. However, their number of passes will be restricted and will depend on the number of SIKS Ph.D. students participating in the course.

Course coordinators

Rineke Verbrugge (RUG), Erman Acar (UvA) and Herke van Hoof (UvA)


The course will take place at hotel Mitland near Utrecht city centre.


Wednesday 20 September 2023: Reinforcement Learning

9:15-9:30 Doors open

9:30-9:45 Course directors: Welcome and Introduction

9:45-10:45 Reinforcement Learning ‘Advanced Basics’ I – Herke van Hoof

10:45-11:15 Coffee break

11:15-12:15 Reinforcement Learning ‘Advanced Basics’ II – Herke van Hoof

12:15-13:30 LUNCH

13:30-14:15 Safe reinforcement learning – Matthijs Spaan

14:15-15:15 Learning policies that generalize in a reliable and systematic way –  Hector Geffner

15:15-15:30 Coffee break

15:30-16:15 Causality and reinforcement learning – Erman Acar

16:15-17:00 Multi-objective learning & preference elicitation – Diederik Roijers

17:30         Dinner

Thursday 21 September 2023: Multi-Agent Reinforcement Learning

9:30-10:30 Basics of multi-agent reinforcement learning, including game theory –  Ann Nowé

10:30 – 11:00 Poster session I

11:00-11:30 Coffee break

11:30-12:15 Communication in multi-agent Reinforcement Learning – Shihan Wang

12:15-13:30 LUNCH

13:30-14:30 Cooperation fallacies: zero-shot coordination and relative overgeneralization –Wendelin Boehmer

14:30-15:00 Poster session II

15:00-15:30 Coffee break

15:30-16:15 RL for ad-hoc collaboration  – Mustafa Mert Çelikok

16:15-17:00 Inverse reinforcement learning and imitation learning in robotics – Jens Kober

17:00-17:05 Closing – course directors


Registration for the course is now closed