Inner product spaces, Kernels, Reproducing kernels, and RKHS. Introductory learning theory and Generalization. Empirical Risk Minimization and Uniform Convergence. Kernel Ridge Regression and Logistic Regression. Optimization and Duality. Margin-based methods and Support vector machines. Kernel PCA and CCA.
Course position and Prerequisites
Course is advanced MSc course in Machine learning, targeted to 1st/2nd year MSc students in Machine Learning and Computer science. Also suitable for PhD studies.
- The course Machine Learning: Basic principles (or equivalent knowledge), strongly recommended
- Python programming skills are recommended (course material will use Python, and there will be an introductory python programming short session)
After attending the course, the student knows the basics of kernels, positive-definiteness, and RKHS. The courses also formally introduces the notions of generalization in machine learning by studying the principle of Empirical Risk Minimization and its consistency. The student knows how convex optimization methods can be used to efficiently train kernel-based and large-scale linear models.It is also discussed how to apply kernel based methods for unsupervised learning such as PCA, and CCA.
- Lectures: Wednesdays 12:15-14:00, lecture hall T1
- Solution sessions: Thursdays 16:15-18:00, TU1 (1017) solution sessions (presenting the solutions for the exercise set) on 31st January, 28th February, 21st March, and 4th April. In these sessions, you can discuss your solutions with the TAs. Attending these sessions is voluntary.
- Programming tutorial : A tutorial for basic programming in python which is required for the course will be given on 17th January, 16:15-18:00, TU1 (1017)
- Assignments will be posted on 16th January, 6th February, 6th March, 20th March. All completed assignments would typically be due within two weeks of the date when the homework assignment is made available
- Exam: 12.04.2019
- Lecturer: Rohit Babbar
- Course assistants: Dr Sandor Szedmak, Viivi Uurtio, Eric Bach
The course can be completed by two alternative ways:
- Exercises (max. 40 points + Bonus points) + Exam (max. 40 points), giving a grade 0..5. Lowest passing points total is 40. 70 points will give the grade of 5.
- Exam only (max. 40 points), giving a grade 0...5. 20 points will give the grade 1, 34 points will give the grade of 5.
The better of the resulting two grades will be taken into account.
Language of Instruction - English
- Lecture slides and exercises are the examined content
- Further references will be provided during respective lectures
- Shawe-Taylor and Cristianini: Kernel Methods for Pattern Analysis, Cambridge University Press, 2004. Available as ebook: http://site.ebrary.com/lib/aalto/detail.action?docID=10131674
- B. Scholkopf, A. Smola: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. http://books.google.fi/books?isbn=0262194759
- Research papers provided during the course (See Materials).
|Date and Time||Type/Location||Content|
|January 9, 12:15||Lecture 1||Basics and Introduction to Kernels|
|January 16, 12:15||Lecture 2||Kernel and Reproducing Kernel Hilbert Space - I|
|January 23, 12:15||Lecture 3||RKHS and Representer Theorem|
|January 30, 12:15||Lecture 4||Introductory Learning theory and Generalization|
|February 6, 12:15||Lecture 5||LearningTheory - II|
|February 13, 12:15||Lecture 6|
Kernel Ridge Regression and Logistic Regression
|February 27, 16:15||Lecture 7||Convexity and Duality|
|February 28, 16:15||Solution Session|
|March 6, 12:15||Lecture 8||Support Vector Machines and |
Large-scale linear Optimization
|March 13, 12:15||Lecture 9||PCA, Clustering and Kernel variants|
|March 14, 16:15||Solution Session || Exercise 2|
|March 20, 12:15||Lecture 10|| CCA and Kernel CCA|
|March 21, 16:15||Tutorial Session|| Exercise 3|
|March 27, 12:15||Lecture||Methods for Large-scale Linear and kernel classification|
|March 28, 16:15||Solution / Tutorial Session|| Exercise 3 / Exercise 4|
|April 3, 12:15||Lecture||Course Recap and Review|
|April 4, 16:15||Solution Session|| Exercise 4|