Lectures
Concept
Lectures will be places of discussion where the current topic is summarized by the lecturer and discussed among all present. The students are expected to prepare by reading given material in advance prior to each lecture.
Arrangements
Course lectures will be given during first and second periods on Tuesdays 14:15-16:00. All lectures will be given over Zoom at link https://aalto.zoom.us/j/63645678644. Please download and install Zoom before the first lecture to attend the course. Lecture recordings will likely be available afterwards, but this cannot be guaranteed due to potential technical issues. The lectures are interactive in nature so that participation is encouraged.
Course lectures will be given by Ville Kyrki (first part) and Joni Pajarinen (second part).
Schedule and Readings
For each lecture starting from the third one, there will be reading materials that the students should study before attending the lecture.
Course arrangements, Overview, Tue 14.9., no readings
Markov decision processes, Tue 21.9., Sutton & Barto, chapters 2-2.3, 2.5-2.6, 3-3.8
RL in discrete domains (value-based RL), Tue 28.9., Sutton&Barto Ch. 5-5.4, 5.6, 6-6.5
Function approximation, Tue 5.10., Sutton&Barto Ch. 9-9.3, 10-10.1
Policy gradient, Tue 12.10., Sutton&Barto, Ch. 13-13.3
Actor-critic, Tue 19.10., Sutton & Barto, Ch. 13.5, 13.7
Towards model-based reinforcement learning: optimal control, Tue 26.10. Platt: Introduction to Linear Quadratic Regulation
Model-based reinforcement learning, Tue 2.11., Sutton & Barto, Ch. 8-8.2
Partially observable MDPs, Tue 16.11., Anthony Cassandra, POMDP tutorial, http://www.pomdp.org/tutorial/, steps from "Brief Introduction to MDPs" until "
General Form of a POMDP solution".
Lecture slides
- FileCourse arrangements FilePDF document
- FileLecture 1: Overview FilePDF document
- FileLecture 2: Markov decision processes FilePDF document
- FileLecture 3: Reinforcement learning FilePDF document
- FileLecture 4: Function approximation FilePDF document
- FileLecture 5: Policy gradient FilePDF document
- FileLecture 6: Actor-critic methods FilePDF document
- FileLecture 7: Optimal control FilePDF document
- FileGuest lecture of Gökhan Alcan FilePDF document
- FileLecture 10: Larger POMDPs FilePDF document
Readings and other materials
- FileExtra slides: Planning in discrete space FilePDF document
Lecture slides from 2020
- FileCourse arrangements FilePDF document
- FileLecture 1: Overview FilePDF document
- FileLecture 2: Markov Decision Processes FilePDF document
- FileLecture 4: Function approximation FilePDF document
- FileLecture 5: Policy gradient FilePDF document
- FileLecture 6: Actor-critic methods FilePDF document
- FileLecture 7: Optimal control FilePDF document
- FileLecture 8: Model-based RL FilePDF document
- FileGuest lecture notes FilePDF document
- FileLecture 10: Large POMDPs FilePDF document
Extra links
Slides from 2019