Topic outline

  • General

    The course is held in person. The announcements will be made in slack, please join slack by clicking this link.

    CS-E407513 - Seminar on Deep Learning (4-6 ECTS)

    Responsible Teacher: Alexander Ilin

    Teachers: Ricardo Falcon Perez, Kate Haitsiukevich, Sam Spilsbury, Nicola Dainese

    Level of the Course: Master's and PhD level

    Teaching Period: I-II

    Description: In this course, we will discuss papers on deep learning and deep reinforcement learning published in this year's ICML, ICLR and NeurIPS. Each student has to

    • present one paper (from the list pre-selected by us),
    • serve as an opponent in one presentation,
    • actively participate in the discussion in slack.

    Learning outcomes:

    • getting familiar with some of the recent papers from the deep learning literature
    • learning to review academic publications
    • learning to present scientific works

    Registration: There are 12 seminar sessions, two papers will be presented in each session. Therefore, we can accept maximum 24 students to the course. Registration for the course will be prioritized by:

    1. Study level (PhD, MSc)
    2. Early registration and selection of the topic.
    3. Grade in the Deep Learning course.

    Prerequisites: CS-E4890 - Deep Learning. This course covers advanced topics and therefore you should be comfortable with the basics.

    Highly recommended: ELEC-E8125 - Reinforcement learning. We usually discuss a few papers on RL. If you do not know the basics of RL, it will be difficult for you to discuss those papers. But active participation in the discussion of non-RL papers should be enough to pass the course.

    Grading: Grade from 0 to 5. The grade will be based on your presentation (30%), opponent speech (20%) and participation in the discussion (50%). Full points for participation can be obtained by contributing to the discussion of 50% (12) of the papers.

    Format of the seminars: The seminars are held in person. Two papers are discussed in each session with a 10-minute break between the papers.

    The format of discussion of one paper:

    • Presentation (max 15 minutes).
    • Opponent speech (max 5 minutes).
    • Discussion moderated by the teachers.

    More information can be found here.

    Discussion in slack: The discussion of the papers will happen primarily in slack. During the seminar sessions, we will only briefly summarize the slack discussion. We will also use slack for course announcements. Please join the slack workspace by following this link.

    IMPORTANT: Students are expected to discuss papers in slack before the start of seminar sessions so that the teachers could select discussion points for the seminars. Asking questions, sharing links to related resources, trying to give answers to questions, raising concerns, pointing out limitations, sharing ideas of possible extensions of the discussed work: all of that counts as participation. Please try to give a critical view on the discussed paper instead of writing a brief summary of it.

  • Seminar schedule

    Session 1 (06.09) Audio

    Session 2 (13.09) Architectures/generalization/insights

    Session 3 (20.09) Architectures/generalization/insights

    Session 4 (27.09) Audio

    Session 5 (04.10) Generative models

    Session 6 (11.10) Language grounding / Graph neural networks

    Session 7 (25.10) Self-supervised learning

    Session 8 (01.11) Graph neural networks

    Session 9 (08.11) Generative models / Audio

    Session 10 (15.11) AGI / Reasoning/compositionality

    Session 11 (22.11) RL

    Session 12 (29.11) RL

  • Information for participants

    Grading
    The grade will be based on your presentation (30%), your service as the opponent (20%) and your participation in the discussions in slack (50%).
    • Seminar attendance without participation in discussions will not affect your grade, so please make sure you contribute to the discussions. We are not going to track seminar attendance.
    • Full points for participation can be obtained by contributing to the discussion of 50% (12) of the papers.
    • If you contribute to the discussion of more than 50% of the papers, you will get extra credit points (up to 6 credits in total).

    Presentation guidelines
    Please read these instructions carefully because the quality of your presentation will affect your grade.
    • The duration of your presentation should be maximum 15 minutes. We will stop presentations after 15 minutes.
    • Try to explain the main idea of the paper instead of copy-pasting paragraphs from the paper page by page. You do not have to use all 15 minutes.
    • Try to give a critical view on the paper: what is the significance of the paper's contributions.
    • Spend minimum time on well-known concepts (such as, for example, GAN or transformer) or concepts that have been discussed earlier in the course. If you are not sure whether a specific topic should be covered in your presentation, you can consult the teachers in the course chat.
    • Do not overload your slides with text. It is better to present figures/illustrations.
    • Please be selective in presenting experimental results. You do not have to present all of them but the most important ones.

    Guidelines for the opponents
    Based on our previous experience, many opponent talks were spending the bulk of the time just summarizing the papers, with very little time spent reviewing and critically engaging with the content. Opponents for the upcoming seminars, please do not include a paper summary in your speech and put more effort in critically reviewing the paper, looking at pros and cons, and overall relevance of the contributions. It may be difficult to cover everything in 5 minutes, so please try to concentrate on what you think are the crucial points. For example, you can focus on the few key contributions you found most important and why they work / don't work and how they compare to other similar work. 

    Course slack
    • Each paper discussed in the course will have its own channel in slack.
    • Please post your questions/comments in the corresponding channel before the start of the corresponding seminar session. Of course, we can continue the discussion after seminar sessions as well.
    • Please try to give a critical view on the discussed paper instead of writing a brief summary of it.

  • Assessments