CS-E4830 - Kernel Methods in Machine Learning, 09.01.2019-25.03.2019
This course space end date is set to 25.03.2019 Search Courses: CS-E4830
Topic outline
-
Contents
Inner product spaces, Kernels, Reproducing kernels, and RKHS. Introductory learning theory and Generalization. Empirical Risk Minimization and Uniform Convergence. Kernel Ridge Regression and Logistic Regression. Optimization and Duality. Margin-based methods and Support vector machines. Kernel PCA and CCA.
Course position and Prerequisites
Course is advanced MSc course in Machine learning, targeted to 1st/2nd year MSc students in Machine Learning and Computer science. Also suitable for PhD studies.
Prerequisite knowledge:
- The course Machine Learning: Basic principles (or equivalent knowledge), strongly recommended
- Python programming skills are recommended (course material will use Python, and there will be an introductory python programming short session)
Learning Outcomes
After attending the course, the student knows the basics of kernels, positive-definiteness, and RKHS. The courses also formally introduces the notions of generalization in machine learning by studying the principle of Empirical Risk Minimization and its consistency. The student knows how convex optimization methods can be used to efficiently train kernel-based and large-scale linear models.It is also discussed how to apply kernel based methods for unsupervised learning such as PCA, and CCA.
Course schedules
- Lectures: Wednesdays 12:15-14:00, lecture hall T1
- Solution sessions: Thursdays 16:15-18:00, TU1 (1017) solution sessions (presenting the solutions for the exercise set) on 31st January, 28th February, 21st March, and 4th April. In these sessions, you can discuss your solutions with the TAs. Attending these sessions is voluntary.
- Programming tutorial : A tutorial for basic programming in python which is required for the course will be given on 17th January, 16:15-18:00, TU1 (1017)
- Assignments will be posted on 16th January, 6th February, 6th March, 20th March. All completed assignments would typically be due within two weeks of the date when the homework assignment is made available
- Exam: 12.04.2019
Course personnel
- Lecturer: Rohit Babbar
- Course assistants: Dr Sandor Szedmak, Viivi Uurtio, Eric Bach
Grading
The course can be completed by two alternative ways:
- Exercises (max. 40 points + Bonus points) + Exam (max. 40 points), giving a grade 0..5. Lowest passing points total is 40. 70 points will give the grade of 5.
- Exam only (max. 40 points), giving a grade 0...5. 20 points will give the grade 1, 34 points will give the grade of 5.
The better of the resulting two grades will be taken into account.
Language of Instruction - English
Course Material
- Lecture slides and exercises are the examined content
- Further references will be provided during respective lectures
Additional reading
- Shawe-Taylor and Cristianini: Kernel Methods for Pattern Analysis, Cambridge University Press, 2004. Available as ebook: http://site.ebrary.com/lib/aalto/detail.action?docID=10131674
- B. Scholkopf, A. Smola: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. http://books.google.fi/books?isbn=0262194759
- Research papers provided during the course (See Materials).
Tentative Schedule Date and Time Type/Location Content January 9, 12:15 Lecture 1 Basics and Introduction to Kernels January 16, 12:15 Lecture 2 Kernel and Reproducing Kernel Hilbert Space - I January 23, 12:15 Lecture 3 RKHS and Representer Theorem January 30, 12:15 Lecture 4 Introductory Learning theory and Generalization February 6, 12:15 Lecture 5 LearningTheory - II February 13, 12:15 Lecture 6 Kernel Ridge Regression and Logistic RegressionFebruary 27, 16:15 Lecture 7 Convexity and Duality February 28, 16:15 Solution Session March 6, 12:15 Lecture 8 Support Vector Machines and
Large-scale linear OptimizationMarch 13, 12:15 Lecture 9 PCA, Clustering and Kernel variants March 14, 16:15 Solution Session Exercise 2 March 20, 12:15 Lecture 10 CCA and Kernel CCA March 21, 16:15 Tutorial Session Exercise 3 March 27, 12:15 Lecture Methods for Large-scale Linear and kernel classification March 28, 16:15 Solution / Tutorial Session Exercise 3 / Exercise 4 April 3, 12:15 Lecture Course Recap and Review April 4, 16:15 Solution Session Exercise 4
- The course Machine Learning: Basic principles (or equivalent knowledge), strongly recommended