Topic outline

  • Contents 

    Inner product spaces, Kernels, Reproducing kernels, and RKHS. Introductory learning theory and Generalization. Empirical Risk Minimization and Uniform Convergence. Kernel Ridge Regression and Logistic Regression. Optimization and Duality. Margin-based methods and Support vector machines. Kernel PCA and  CCA. 

    Course position and Prerequisites 

    Course is advanced MSc course in Machine learning, targeted to 1st/2nd year MSc students in Machine Learning and Computer science.  Also suitable for PhD studies.

    Prerequisite knowledge:

    • The course Machine Learning: Basic principles (or equivalent knowledge), strongly recommended
    • Python programming skills are recommended (course material will use Python, and there will be an introductory python programming short session)

    Learning Outcomes 

    After attending the course, the student knows the basics of kernels,  positive-definiteness, and RKHS. The courses also formally introduces the notions of generalization in machine learning by studying the principle of Empirical Risk Minimization and its consistency. The student knows how convex optimization methods can be used to efficiently train kernel-based and large-scale linear models.It is also discussed how to apply kernel based methods for unsupervised learning such as PCA, and CCA.

    Course schedules

    • Lectures: Wednesdays 12:15-14:00, lecture hall T1
    • Solution sessions: Thursdays 16:15-18:00, TU1 (1017) solution sessions (presenting the solutions for the exercise set) on 31st January, 28th February, 21st March, and 4th April. In these sessions, you can discuss your solutions with the TAs. Attending these sessions is voluntary.
    • Programming tutorial : A tutorial for basic programming in python which is required for the course will be given on 17th January, 16:15-18:00, TU1 (1017)
    • Assignments will be posted on 16th January, 6th February, 6th March, 20th March. All completed assignments would typically be due within two weeks of the date when the homework assignment is made available
    • Exam: 12.04.2019

    Course personnel

    • Lecturer: Rohit Babbar
    • Course assistants: Dr Sandor Szedmak, Viivi Uurtio, Eric Bach


    Grading 

    The course can be completed by two alternative ways:

    • Exercises (max. 40 points + Bonus points) + Exam  (max. 40 points), giving a grade 0..5. Lowest passing points total is 40. 70 points will give the grade of 5.
    • Exam only (max. 40 points), giving a grade 0...5. 20 points will give the grade 1, 34 points will give the grade of 5.

    The better of the resulting two grades will be taken into account.

    Language of InstructionEnglish

    Course Material 

    • Lecture slides and exercises are the examined content
    • Further references will be provided during respective lectures 

    Additional reading

      • Research papers provided during the course (See Materials).


      Tentative Schedule
      Date and TimeType/LocationContent
      January 9, 12:15Lecture 1 Basics and Introduction to Kernels
      January 16, 12:15Lecture 2 Kernel and Reproducing Kernel Hilbert Space - I
      January 23, 12:15Lecture 3 RKHS and Representer Theorem
      January 30, 12:15Lecture 4 Introductory Learning theory and Generalization
      February 6, 12:15Lecture 5 LearningTheory - II
      February 13, 12:15Lecture 6 
      Kernel Ridge Regression and Logistic Regression
      February 27, 16:15
      Lecture 7Convexity and Duality
      February 28, 16:15
      Solution Session
      March 6, 12:15Lecture 8Support Vector Machines and 
      Large-scale linear Optimization
      March 13, 12:15Lecture 9PCA, Clustering and Kernel variants
      March 14, 16:15
      Solution Session
       Exercise 2
      March 20, 12:15Lecture 10 CCA and Kernel CCA
      March 21, 16:15Tutorial Session Exercise 3
      March 27, 12:15LectureMethods for Large-scale Linear and kernel classification
      March 28, 16:15
      Solution / Tutorial Session
       Exercise 3 / Exercise 4
      April 3, 12:15Lecture Course Recap and Review
      April 4, 16:15Solution Session
       Exercise 4