The tenth meeting took place on the 24th of July at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Chris Pasek was speaking about a recent OpenAI & DeepMind paper: Deep reinforcement learning from human preferences. The recording (in Polish) is available below.

## IX: Convergence of Instrumental Goals

The ninth meeting took place on the 12th of June at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Chris Pasek was speaking about a formalization of an argument for the ubiquity of certain goals in superintelligent agents, basing on Formalizing Convergent Instrumental Goals. The recording (in Polish) is available below.

## VIII: Reflective Oracles

The eigth meeting took place on the 5th of June at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Tomek Kisielewski was speaking about reflective oracles, basing on Reflective Oracles: A Foundation for Classical Game Theory and Reflective Variants of Solomonoff Induction and AIXI. The recording (in Polish) is available below.

## VII: AI Policy

The seventh meeting took place on the 22nd of May at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Marcin Woźniak was speaking about AI policy, partly basing on the papers Policy Desiderata in the Development of Machine Superintelligence and Strategic Implications of Openness in AI Development. The recording (in Polish) is available below and the mind map here.

## VI: Safely Interruptible Agents

The sixth meeting took place on the 15th of May at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Damian Leśniak was speaking about safely turning off agents in the reinforcement learning framework based on the paper Safely Interruptible Agents. The recording (in Polish) is available below.

## V: An Introduction to Reinforcement Learning

The fifth meeting took place on the 8th of May at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Igor Sieradzki talked about the basic ideas of reinforcement learning. The recording (in Polish) is available below, and the slides here.

## IV: A Short Survey of Decision Theory

The fourth meeting took place on the 24th of April at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Tomek Kisielewski was speaking about the current state of decision theory and its future directions based on the paper Towards Idealized Decision Theory. The recording (in Polish) is available below.

## III: Corrigibility of Agents

The third meeting took place on the 10th of April at 14:15 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Krzysztof Galias was speaking about in-use modification of agents based on the paper Corrigibility. The recording (in Polish) is available below, although it cuts off a bit too early.

## II: Cooperation in the Open-source Prisoner’s Dilemma

The second meeting took place on the 3rd of April at 14:00 in room 0006 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Tomek Kisielewski was speaking about cooperation between agents due to an application of Löb’s theorem based on the papers Robust Cooperation in the Prisoner’s Dilemma: Program Equilibrium via Provability Logic and Parametric Bounded Löb’s Theorem and Robust Cooperation of Bounded Agents. The recording (in Polish) is available below.

## I: Problems in AI Safety

The first meeting took place on the 27th of March at 14:00 in room 0116 at the Faculty of Mathematics and Computer Science of the Jagiellonian University. Krzysztof Galias presented an introduction to problems in AI safety based on the papers Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda and Concrete Problems in AI Safety. The recording (in Polish) is available below.