ELEC-E8125 - Reinforcement learning D, Lecture, 13.9.2021-8.12.2021

Osion kuvaus

• Lectures
  Concept

  Lectures will be places of discussion where the current topic is summarized by the lecturer and discussed among all present. The students are expected to prepare by reading given material in advance prior to each lecture.
  

  Arrangements
  

  Course lectures will be given during first and second periods on Tuesdays 14:15-16:00. All lectures will be given over Zoom at link https://aalto.zoom.us/j/63645678644. Please download and install Zoom before the first lecture to attend the course. Lecture recordings will likely be available afterwards, but this cannot be guaranteed due to potential technical issues. The lectures are interactive in nature so that participation is encouraged.

  Course lectures will be given by Ville Kyrki (first part) and Joni Pajarinen (second part).
  

  Schedule and Readings
  

  For each lecture starting from the third one, there will be reading materials that the students should study before attending the lecture.

  Course arrangements, Overview, Tue 14.9., no readings
  

  Markov decision processes, Tue 21.9., Sutton & Barto, chapters 2-2.3, 2.5-2.6, 3-3.8

  RL in discrete domains (value-based RL), Tue 28.9., Sutton&Barto Ch. 5-5.4, 5.6, 6-6.5

  Function approximation, Tue 5.10., Sutton&Barto Ch. 9-9.3, 10-10.1

  Policy gradient, Tue 12.10.,  Sutton&Barto, Ch. 13-13.3
  

  Actor-critic, Tue 19.10., Sutton & Barto, Ch. 13.5, 13.7
  

  Towards model-based reinforcement learning: optimal control, Tue 26.10. Platt: Introduction to Linear Quadratic Regulation
  

  Model-based reinforcement learning, Tue 2.11., Sutton & Barto, Ch. 8-8.2

  Guest lectures Tue 9.11.: Safety and constraints (Gökhan Alcal), Entropy Regularization in Reinforcement Learning (Riad Akrour)

  
  

  Partially observable MDPs, Tue 16.11., Anthony Cassandra, POMDP tutorial, http://www.pomdp.org/tutorial/, steps from "Brief Introduction to MDPs" until " General Form of a POMDP solution".
  

  Large POMDPs, Tue 23.11.

  
  

  Project show!, to be confirmed

  • Valitse aktiviteetti Lecture slides

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi

    Lecture slides

  • Valitse aktiviteetti Course arrangements

    

    Course arrangements Tiedosto PDF
  • Valitse aktiviteetti Lecture 1: Overview

    

    Lecture 1: Overview Tiedosto PDF
  • Valitse aktiviteetti Lecture 2: Markov decision processes

    

    Lecture 2: Markov decision processes Tiedosto PDF
  • Valitse aktiviteetti Lecture 3: Reinforcement learning

    

    Lecture 3: Reinforcement learning Tiedosto PDF
  • Valitse aktiviteetti Lecture 3 recording (Aalto login only)

    

    Lecture 3 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 4: Function approximation

    

    Lecture 4: Function approximation Tiedosto PDF
  • Valitse aktiviteetti Lecture 4 recording (Aalto login only)

    

    Lecture 4 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 5: Policy gradient

    

    Lecture 5: Policy gradient Tiedosto PDF
  • Valitse aktiviteetti Lecture 5 recording (Aalto login only)

    

    Lecture 5 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 6: Actor-critic methods

    

    Lecture 6: Actor-critic methods Tiedosto PDF
  • Valitse aktiviteetti Lecture 6 recording (Aalto login only)

    

    Lecture 6 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 7: Optimal control

    

    Lecture 7: Optimal control Tiedosto PDF
  • Valitse aktiviteetti Lecture 7 recording (Aalto login only)

    

    Lecture 7 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 8: Model-based reinforcement learning

    

    Lecture 8: Model-based reinforcement learning Tiedosto PDF
  • Valitse aktiviteetti Lecture 8 recording (Aalto login only)

    

    Lecture 8 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Guest lecture of Gökhan Alcan

    

    Guest lecture of Gökhan Alcan Tiedosto PDF
  • Valitse aktiviteetti Guest lecture of Gökhan Alcan recording (Aalto login only)

    

    Guest lecture of Gökhan Alcan recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Guest Lecture on Entropy Regularization in Reinforcement Learning (Riad Akrour)

    

    Guest Lecture on Entropy Regularization in Reinforcement Learning (Riad Akrour) Tiedosto PDF
  • Valitse aktiviteetti Guest Lecture on Entropy Regularization in Reinforcement Learning recording (Aalto login only)

    

    Guest Lecture on Entropy Regularization in Reinforcement Learning recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 9: Partially observable Markov decision processes (POMDPs)

    

    Lecture 9: Partially observable Markov decision processes (POMDPs) Tiedosto PDF
  • Valitse aktiviteetti Lecture 9 recording (Aalto login only)

    

    Lecture 9 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 10: Larger POMDPs

    

    Lecture 10: Larger POMDPs Tiedosto PDF
  • Valitse aktiviteetti Lecture 10 recording (Aalto login only)

    

    Lecture 10 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Readings and other materials

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi

    Readings and other materials
    

  • Valitse aktiviteetti Sutton & Barto, Reinforcement Learning: An introduction, 2nd ed.

    

    Sutton & Barto, Reinforcement Learning: An introduction, 2nd ed. Verkko-osoite

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi
  • Valitse aktiviteetti Extra slides: Planning in discrete space

    

    Extra slides: Planning in discrete space Tiedosto PDF

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi
  • Valitse aktiviteetti Platt, Introduction to Linear Quadratic Regulation

    

    Platt, Introduction to Linear Quadratic Regulation Verkko-osoite

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi
  • Valitse aktiviteetti Reinforcement learning course at UCL

    

    Reinforcement learning course at UCL Verkko-osoite
  • Valitse aktiviteetti Deep reinforcement learning course at UC Berkeley

    

    Deep reinforcement learning course at UC Berkeley Verkko-osoite
  • Valitse aktiviteetti LaValle, Planning Algorithms

    

    LaValle, Planning Algorithms Verkko-osoite

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi
  • Valitse aktiviteetti Lecture slides from 2020

    Suoritus

    Opiskelijoiden täytyy

    

    Merkitse tehdyksi

    Lecture slides from 2020
    

  • Valitse aktiviteetti Course arrangements

    

    Course arrangements Tiedosto PDF
  • Valitse aktiviteetti Lecture 1: Overview

    

    Lecture 1: Overview Tiedosto PDF
  • Valitse aktiviteetti Lecture 1 recording (requires Aalto account)

    

    Lecture 1 recording (requires Aalto account) Verkko-osoite
  • Valitse aktiviteetti Lecture 2: Markov Decision Processes

    

    Lecture 2: Markov Decision Processes Tiedosto PDF
  • Valitse aktiviteetti Lecture 2 recording (Aalto login only)

    

    Lecture 2 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 3: Reinforcement learning in discrete domains

    

    Lecture 3: Reinforcement learning in discrete domains Tiedosto PDF
  • Valitse aktiviteetti Lecture 3 recording (Aalto login only)

    

    Lecture 3 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 4: Function approximation

    

    Lecture 4: Function approximation Tiedosto PDF
  • Valitse aktiviteetti Lecture 5: Policy gradient

    

    Lecture 5: Policy gradient Tiedosto PDF
  • Valitse aktiviteetti Lecture 5 recording (Aalto login only)

    

    Lecture 5 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 6: Actor-critic methods

    

    Lecture 6: Actor-critic methods Tiedosto PDF
  • Valitse aktiviteetti Lecture 6 recording (Aalto login only)

    

    Lecture 6 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 7: Optimal control

    

    Lecture 7: Optimal control Tiedosto PDF
  • Valitse aktiviteetti Lecture 7 recording (Aalto login only)

    

    Lecture 7 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 8: Model-based RL

    

    Lecture 8: Model-based RL Tiedosto PDF
  • Valitse aktiviteetti Lecture 8 recording, only partial available (Aalto login only)

    

    Lecture 8 recording, only partial available (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Guest lecture: Safety and optimal control (Gökhan Alcan)

    

    Guest lecture: Safety and optimal control (Gökhan Alcan) Tiedosto PDF
  • Valitse aktiviteetti Guest lecture recording (Aalto login only)

    

    Guest lecture recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Guest lecture notes

    

    Guest lecture notes Tiedosto PDF
  • Valitse aktiviteetti Lecture 9: Partially Observable Markov Decision Processes

    

    Lecture 9: Partially Observable Markov Decision Processes Tiedosto PDF
  • Valitse aktiviteetti Lecture 9 recording (Aalto login only)

    

    Lecture 9 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Lecture 10: Large POMDPs

    

    Lecture 10: Large POMDPs Tiedosto PDF
  • Valitse aktiviteetti Lecture 10 recording (Aalto login only)

    

    Lecture 10 recording (Aalto login only) Verkko-osoite
  • Valitse aktiviteetti Extra links

    Extra links
    

  • Valitse aktiviteetti Model-based reinforcement learning tutorial (ICML 2020)

    

    Model-based reinforcement learning tutorial (ICML 2020) Verkko-osoite
  • Valitse aktiviteetti Bradberry, "Introduction to Monte Carlo Tree Search"

    

    Bradberry, "Introduction to Monte Carlo Tree Search" Verkko-osoite
  • Valitse aktiviteetti Deep Reinforcement Learning: Pong from Pixels

    

    Deep Reinforcement Learning: Pong from Pixels Verkko-osoite
  • Valitse aktiviteetti AlphaGo: using machine learning to master the ancient game of Go

    

    AlphaGo: using machine learning to master the ancient game of Go Verkko-osoite
  • Valitse aktiviteetti Andrew Ng, "Nuts and Bolts of Applying Deep Learning" (video)

    

    Andrew Ng, "Nuts and Bolts of Applying Deep Learning" (video) Verkko-osoite
  • Valitse aktiviteetti Juliani, Simple Reinforcement Learning with Tensorflow

    

    Juliani, Simple Reinforcement Learning with Tensorflow Verkko-osoite
  • Valitse aktiviteetti Slides from 2019

    Slides from 2019
    

  • Valitse aktiviteetti Guest lecture: Deep reinforcement learning in robotics

    

    Guest lecture: Deep reinforcement learning in robotics Tiedosto PDF