Course: ELEC-E8125 - Reinforcement learning D, Lecture, 13.9.2021-8.12.2021, Topic: Lectures

Home Schools Course feedback Service Links Intelliboard

This course space end date is set to 08.12.2021 Search Courses: ELEC-E8125

Topic outline

Lectures
Concept

Lectures will be places of discussion where the current topic is summarized by the lecturer and discussed among all present. The students are expected to prepare by reading given material in advance prior to each lecture.

Arrangements

Course lectures will be given during first and second periods on Tuesdays 14:15-16:00. All lectures will be given over Zoom at link https://aalto.zoom.us/j/63645678644. Please download and install Zoom before the first lecture to attend the course. Lecture recordings will likely be available afterwards, but this cannot be guaranteed due to potential technical issues. The lectures are interactive in nature so that participation is encouraged.
Course lectures will be given by Ville Kyrki (first part) and Joni Pajarinen (second part).

Schedule and Readings

For each lecture starting from the third one, there will be reading materials that the students should study before attending the lecture.

Course arrangements, Overview, Tue 14.9., no readings
Markov decision processes, Tue 21.9., Sutton & Barto, chapters 2-2.3, 2.5-2.6, 3-3.8
RL in discrete domains (value-based RL), Tue 28.9., Sutton&Barto Ch. 5-5.4, 5.6, 6-6.5
Function approximation, Tue 5.10., Sutton&Barto Ch. 9-9.3, 10-10.1
Policy gradient, Tue 12.10., Sutton&Barto, Ch. 13-13.3
Actor-critic, Tue 19.10., Sutton & Barto, Ch. 13.5, 13.7
Towards model-based reinforcement learning: optimal control, Tue 26.10. Platt: Introduction to Linear Quadratic Regulation
Model-based reinforcement learning, Tue 2.11., Sutton & Barto, Ch. 8-8.2
Guest lectures Tue 9.11.: Safety and constraints (Gökhan Alcal), Entropy Regularization in Reinforcement Learning (Riad Akrour)

Partially observable MDPs, Tue 16.11., Anthony Cassandra, POMDP tutorial, http://www.pomdp.org/tutorial/, steps from "Brief Introduction to MDPs" until " General Form of a POMDP solution".

Large POMDPs, Tue 23.11.

Project show!, to be confirmed
- Lecture slides
- Course arrangements File
  
  PDF document
- Lecture 1: Overview File
  
  PDF document
- Lecture 2: Markov decision processes File
  
  PDF document
- Lecture 3: Reinforcement learning File
  
  PDF document
- Lecture 3 recording (Aalto login only) URL
- Lecture 4: Function approximation File
  
  PDF document
- Lecture 4 recording (Aalto login only) URL
- Lecture 5: Policy gradient File
  
  PDF document
- Lecture 5 recording (Aalto login only) URL
- Lecture 6: Actor-critic methods File
  
  PDF document
- Lecture 6 recording (Aalto login only) URL
- Lecture 7: Optimal control File
  
  PDF document
- Lecture 7 recording (Aalto login only) URL
- Lecture 8: Model-based reinforcement learning File
  
  PDF document
- Lecture 8 recording (Aalto login only) URL
- Guest lecture of Gökhan Alcan File
  
  PDF document
- Guest lecture of Gökhan Alcan recording (Aalto login only) URL
- Guest Lecture on Entropy Regularization in Reinforcement Learning (Riad Akrour) File
  
  PDF document
- Guest Lecture on Entropy Regularization in Reinforcement Learning recording (Aalto login only) URL
- Lecture 9: Partially observable Markov decision processes (POMDPs) File
  
  PDF document
- Lecture 9 recording (Aalto login only) URL
- Lecture 10: Larger POMDPs File
  
  PDF document
- Lecture 10 recording (Aalto login only) URL
- Readings and other materials
- Sutton & Barto, Reinforcement Learning: An introduction, 2nd ed. URL
- Extra slides: Planning in discrete space File
  
  PDF document
- Platt, Introduction to Linear Quadratic Regulation URL
- Reinforcement learning course at UCL URL
- Deep reinforcement learning course at UC Berkeley URL
- LaValle, Planning Algorithms URL
- Lecture slides from 2020
- Course arrangements File
  
  PDF document
- Lecture 1: Overview File
  
  PDF document
- Lecture 1 recording (requires Aalto account) URL
- Lecture 2: Markov Decision Processes File
  
  PDF document
- Lecture 2 recording (Aalto login only) URL
- Lecture 3: Reinforcement learning in discrete domains File
  
  PDF document
- Lecture 3 recording (Aalto login only) URL
- Lecture 4: Function approximation File
  
  PDF document
- Lecture 5: Policy gradient File
  
  PDF document
- Lecture 5 recording (Aalto login only) URL
- Lecture 6: Actor-critic methods File
  
  PDF document
- Lecture 6 recording (Aalto login only) URL
- Lecture 7: Optimal control File
  
  PDF document
- Lecture 7 recording (Aalto login only) URL
- Lecture 8: Model-based RL File
  
  PDF document
- Lecture 8 recording, only partial available (Aalto login only) URL
- Guest lecture: Safety and optimal control (Gökhan Alcan) File
  
  PDF document
- Guest lecture recording (Aalto login only) URL
- Guest lecture notes File
  
  PDF document
- Lecture 9: Partially Observable Markov Decision Processes File
  
  PDF document
- Lecture 9 recording (Aalto login only) URL
- Lecture 10: Large POMDPs File
  
  PDF document
- Lecture 10 recording (Aalto login only) URL
- Extra links
- Model-based reinforcement learning tutorial (ICML 2020) URL
- Bradberry, "Introduction to Monte Carlo Tree Search" URL
- Deep Reinforcement Learning: Pong from Pixels URL
- AlphaGo: using machine learning to master the ancient game of Go URL
- Andrew Ng, "Nuts and Bolts of Applying Deep Learning" (video) URL
- Juliani, Simple Reinforcement Learning with Tensorflow URL
- Slides from 2019
- Guest lecture: Deep reinforcement learning in robotics File
  
  PDF document