CS-E407513 - Special Course in Machine Learning, Data Science and Artificial Intelligence D: Seminar on Deep Learning 2022, Lectures, 6.9.2022-29.11.2022
This course space end date is set to 29.11.2022 Search Courses: CS-E407513
Topic outline
-
The course is held in person. The announcements will be made in slack, please join slack by clicking this link.
CS-E407513 - Seminar on Deep Learning (4-6 ECTS)
Responsible Teacher: Alexander Ilin
Teachers: Ricardo Falcon Perez, Kate Haitsiukevich, Sam Spilsbury, Nicola Dainese
Level of the Course: Master's and PhD level
Teaching Period: I-II
Description: In this course, we will discuss papers on deep learning and deep reinforcement learning published in this year's ICML, ICLR and NeurIPS. Each student has to
- present one paper (from the list pre-selected by us),
- serve as an opponent in one presentation,
- actively participate in the discussion in slack.
Learning outcomes:
- getting familiar with some of the recent papers from the deep learning literature
- learning to review academic publications
- learning to present scientific works
Registration: There are 12 seminar sessions, two papers will be presented in each session. Therefore, we can accept maximum 24 students to the course. Registration for the course will be prioritized by:
- Study level (PhD, MSc)
- Early registration and selection of the topic.
- Grade in the Deep Learning course.
Prerequisites: CS-E4890 - Deep Learning. This course covers advanced topics and therefore you should be comfortable with the basics.
Highly recommended: ELEC-E8125 - Reinforcement learning. We usually discuss a few papers on RL. If you do not know the basics of RL, it will be difficult for you to discuss those papers. But active participation in the discussion of non-RL papers should be enough to pass the course.
Grading: Grade from 0 to 5. The grade will be based on your presentation (30%), opponent speech (20%) and participation in the discussion (50%). Full points for participation can be obtained by contributing to the discussion of 50% (12) of the papers.
Format of the seminars: The seminars are held in person. Two papers are discussed in each session with a 10-minute break between the papers.
The format of discussion of one paper:
- Presentation (max 15 minutes).
- Opponent speech (max 5 minutes).
- Discussion moderated by the teachers.
More information can be found here.
Discussion in slack: The discussion of the papers will happen primarily in slack. During the seminar sessions, we will only briefly summarize the slack discussion. We will also use slack for course announcements. Please join the slack workspace by following this link.
IMPORTANT: Students are expected to discuss papers in slack before the start of seminar sessions so that the teachers could select discussion points for the seminars. Asking questions, sharing links to related resources, trying to give answers to questions, raising concerns, pointing out limitations, sharing ideas of possible extensions of the discussed work: all of that counts as participation. Please try to give a critical view on the discussed paper instead of writing a brief summary of it. -
Session 1 (06.09) Audio
- It’s Raw! Audio Generation with State-Space Models
presenter: Teodors Kerimovs
opponent: - Style Transfer of Audio Effects with Differentiable Signal Processing
presenter: Otto Mikkonen
opponent:
Session 2 (13.09) Architectures/generalization/insights
- Why do tree-based models still outperform deep learning on tabular data?
presenter: Hiroshi Doyu
opponent: Yaroslav Getman - Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
presenter: Aayush Kucheria
opponent: Bernard Spiegl
Session 3 (20.09) Architectures/generalization/insights
- Predicting Out-of-Domain Generalization with Local Manifold Smoothness
presenter: Jesse Hamalainen
opponent: Bryn Louise - Epistemic Neural Networks
presenter: Aleksi Maunu
opponent: Eloi Moliner
Session 4 (27.09) Audio
- Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance
presenter: Antoni Jankowski
opponent: Alexandru Dumitrescu - NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates
presenter: Eloi Moliner
opponent: Daniel Carter
Session 5 (04.10) Generative models
- Structured Denoising Diffusion Models in Discrete State Spaces
presenter: Alexandru Dumitrescu
opponent: Antoni Jankowski - High-Resolution Image Synthesis with Latent Diffusion Models
presenter: Hyunkyung Choo
opponent: My Linh Nguyen
Session 6 (11.10) Language grounding / Graph neural networks
- Flamingo: a Visual Language Model for Few-Shot Learning
presenter: My Linh Nguyen
opponent: Christian Montecchiani - Pure Transformers are Powerful Graph Learners
presenter: Maxim Smirnov
opponent: Hiroshi Doyu
Session 7 (25.10) Self-supervised learning
- data2vec: A General Framework for Self-Supervised Learning in Speech; Vision and Language
presenter: Yaroslav Getman
opponent: Jesse Hamalainen - Masked Siamese Networks for Label-Efficient Learning
presenter: Katja Voskoboinik
opponent: Aleksi Maunu
Session 8 (01.11) Graph neural networks
- Equivariant Diffusion for Molecule Generation in 3D
presenter: Severi Rissanen
opponent: Riccardo Mereu - Constraint-based graph network simulator
presenter: Bryn Louise
opponent: Katja Voskoboinik
Session 9 (08.11) Generative models / Audio
- Label-Efficient Semantic Segmentation with Diffusion Models
presenter: Hua Huang
opponent: Aayush Kucheria - Chunked Autoregressive GAN for Conditional Waveform Synthesis
presenter: Filip Zawadka
opponent: Hyunkyung Choo
Session 10 (15.11) AGI / Reasoning/compositionality
- Gato: A Generalist Agent
presenter: Bernard Spiegl
opponent: Anton Naumov - Learning Iterative Reasoning through Energy Minimization
presenter: Riccardo Mereu
opponent: Maximilian Krahn
Session 11 (22.11) RL
- Offline RL Policies Should Be Trained to be Adaptive
presenter: Christian Montecchiani
opponent: Filip Zawadka - Decision Transformer: Reinforcement Learning via Sequence Modeling
presenter: Daniel Carter
opponent: Hua Huang
Session 12 (29.11) RL
- BYOL-Explore: Exploration by Bootstrapped Prediction
presenter: Anton Naumov
opponent: Maxim Smirnov - Transformers are Sample Efficient World Models
presenter: Maximilian Krahn
opponent: Severi Rissanen
- It’s Raw! Audio Generation with State-Space Models
-
Grading
The grade will be based on your presentation (30%), your service as the opponent (20%) and your participation in the discussions in slack (50%).
- Seminar attendance without participation in discussions will not affect your grade, so please make sure you contribute to the discussions. We are not going to track seminar attendance.
- Full points for participation can be obtained by contributing to the discussion of 50% (12) of the papers.
- If you contribute to the discussion of more than 50% of the papers, you will get extra credit points (up to 6 credits in total).
Presentation guidelines
Please read these instructions carefully because the quality of your presentation will affect your grade.- The duration of your presentation should be maximum 15 minutes. We will stop presentations after 15 minutes.
- Try to explain the main idea of the paper instead of copy-pasting paragraphs from the paper page by page. You do not have to use all 15 minutes.
- Try to give a critical view on the paper: what is the significance of the paper's contributions.
- Spend minimum time on well-known concepts (such as, for example, GAN or transformer) or concepts that have been discussed earlier in the course. If you are not sure whether a specific topic should be covered in your presentation, you can consult the teachers in the course chat.
- Do not overload your slides with text. It is better to present figures/illustrations.
- Please be selective in presenting experimental results. You do not have to present all of them but the most important ones.
Guidelines for the opponents
Based on our previous experience, many opponent talks were spending the bulk of the time just summarizing the papers, with very little time spent reviewing and critically engaging with the content. Opponents for the upcoming seminars, please do not include a paper summary in your speech and put more effort in critically reviewing the paper, looking at pros and cons, and overall relevance of the contributions. It may be difficult to cover everything in 5 minutes, so please try to concentrate on what you think are the crucial points. For example, you can focus on the few key contributions you found most important and why they work / don't work and how they compare to other similar work.Course slack
- Each paper discussed in the course will have its own channel in slack.
- Please post your questions/comments in the corresponding channel before the start of the corresponding seminar session. Of course, we can continue the discussion after seminar sessions as well.
- Please try to give a critical view on the discussed paper instead of writing a brief summary of it.
-