CS-E4890 Deep Learning (5 ECTS), 2017.
Responsible Teacher: Jyri Kivinen
Lecturers: Antti Keurulainen [AK], Jyri Kivinen [JK], Jorma Laaksonen [JL]
Assistants: Atli Buse, Kunal Ghosh, Pashupati Hegde, Siddharth Ramchandran, Jaakko Reinvall
Level of the Course: Master's level
Teaching Period: II (Autumn)
Workload: 5 ECTS, so that there are 2 lectures per week (1h 30 min total each), one (primarily computer) exercise session per week (1 h 30 min total each), and the rest for studying the course material, doing exercises, the mini-project, and the examination.
Learning Outcomes: Understanding of the general principles of neural networks and deep learning, and the central neural network and deep learning methods discussed in the course. After the course, you should be able to apply them to real-world data sets.
Content: The course takes elements (e.g. contains material) from prior 'Machine Learning and Neural Networks'-courses and 'Deep Learning'-special courses. Topics will include feed-forward neural networks, convolutional neural networks, optimization, regularization, sequence modelling, practical methodologies, applications, linear factors models, and autoencoders. Implementing algorithms on a computer are a part of the course and the programming language is Python. Python-based softwares that allow for symbolic differentiation will also be used, Theano will be recommended, supported, and there will be teaching on it, along with some Python for data analysis.
Arrangements: Lectures, exercise sessions, an examination.
Times and locations of weekly lectures (held 30.10-5.12.2017):
- Mon 12:15-14, R030/T1
- Tue 14:15-16, R030/T1
Times and locations of weekly computer class sessions (held 2.11.-8.12.2017):
- Thu 10:15-12:00, R017/Maari-C
- Thu 16:15-18:00, R017/Maari-C
- Thu 16:15-18:00, R030/T1 (bring your own computer)
- Fri 12:15-14:00, R001/Y342a
- Fri 14:15-16:00, R001/Y342a
Times and locations of examinations:
- Fri 15.12.2017 13:00-16:00; R030/T1 [those with surnames with first letter within A-N], R001/D [those with surnames with first letter within O-Z]; (default exam)
- Mon 12.02.2018 09:00-12:00; R030/T1
- Mon 21.05.2018 09:00-12:00; R030/T1
Assessment Methods and Criteria: Exercise sets, mini-project, an examination.
Exercise set deadlines: six sets of weekly exercises, each with submission deadline on Wednesday (at 23:55) of the (respectively) following week.
The examination dates are given above.
Study Material: Material produced for the course such as lecture slides, external material. The external material will include the 'Deep Learning'-book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, published by MIT Press in 2016 (online version freely available at http://www.deeplearningbook.org), Python- and Theano-related material.
Substitutes for Courses: 'CS-E4810 Machine Learning and Neural Networks'
Prerequisites: 'CS-E3210 Machine Learning: Basic Principles'-course (can be taken during the same autumn) or knowledge, skills and experience equivalent to that obtained from completing the course; basic courses in mathematics and probability; Python programming basics.
Grading Scale: 0-5
Language of Instruction: English
- First week of teaching (week 44, 30.10-5.11.)
- Monday [30.10.]: Lecture 1 [JL]: Welcome to the course, course overview, deep learning overview, ML-recap.
- Tuesday [31.10.]: Lecture 2 [JL]: Feed-forward neural networks I, Theano.
- Thursday-Friday: Computer class exercise sessions, exercise set 1.
- Second week of teaching (week 45, 6.11.-12.11.)
- Monday [6.11.]: Lecture 3 [JL]: Feed-forward neural networks II, Practical methodologies I.
- Tuesday [7.11.]: Lecture 4 [JK]: Convolutional neural networks I, Practical methodologies II.
- Wednesday [8.11.]: Exercise set 1 submission deadline (at 23:55).
- Thursday-Friday: Computer class exercise sessions, exercise set 2.
- Third week of teaching (week 46, 13.11.-19.11.)
- Monday [13.11.]: Lecture 5 [JK]: Convolutional neural networks II.
- Tuesday [14.11]: Lecture 6 [AK]: Optimization for training deep models.
- Wednesday [15.11.]: Exercise set 2 submission deadline (at 23:55).
- Thursday-Friday: Computer class exercise sessions, exercise set 3.
- Fourth week of teaching (week 47, 20.11.-26.11.):
- Monday [20.11.]: Lecture 7 [AK]: Regularization for deep learning.
- Tuesday [21.11.]: Lecture 8 [AK]: Sequence modelling, recurrent neural networks.
- Wednesday [22.11.]: Exercise set 3 submission deadline (at 23:55).
- Thursday-Friday: Computer class exercise sessions, exercise set 4.
- Fifth week of teaching (week 48, 27.11.-3.12.):
- Monday [27.11.]: Lecture 9 [JK]: Linear factor models, autoencoders.
- Tuesday [28.11.]: Lecture 10 [JK]: Introduction to advanced probabilistic models and methods I (focus on variational autoencoders).
- Wednesday [29.11.]: Exercise set 4 submission deadline (at 23:55).
- Thursday-Friday: Computer class exercise sessions, exercise set 5.
- Sixth (last) week of teaching (week 49, 4.12.-10.12.):
- Monday [4.12.]: Lecture 11 [JK]: Introduction to advanced probabilistic models and methods II (focus on Boltzmann machine-based approaches).
- Tuesday [5.12.]: Lecture 12 [JL]: Applications: Computer vision, natural language processing, other.
- Wednesday [6.12.]: Exercise set 5 submission deadline (at 23:55).
- Thursday-Friday: Computer class exercise sessions, exercise set 6.
- Wednesday 13.12.2017: Exercise set 6 submission deadline (at 23:55).
- Friday 15.12.2017: first possibility to do the exam.
- Wednesday 31.1.2018: deadline of the mini-project (at 23:55).
- Monday 12.2.2018: second possibility to do the exam.
- Monday 21.5.2018: third possibility to do the exam.
- Friday 7.9.2018; fourth possibility to do the exam.
- Wednesday 24.10.2018; fifth (and last) possibility to do the exam.