Topic outline

  • General

    Lecturers:
    Prof. Simo Särkkä (simo.sarkka@aalto.fi).

    Co-lecturers / assistants:
    M.Sc. Filip Tronarp (filip.tronarp@aalto.fi)

    Please add "ELEC-E8105" to subject when sending mail concerning the course.

    Learning Outcomes:
    The student understands the Bayesian basis of estimation in non-linear and non-Gaussian systems. The student understands the principles behind approximate filters and smoothers, and is able to use them in practice. Student knows how to estimate parameters online and offline in non-linear systems.

    Contents:
    Statistical modeling and estimation of non-linear and non-Gaussian systems. Bayesian filtering and smoothing theory. Extended Kalman filtering and smoothing, sigma-point and unscented filtering and smoothing, sequential Monte Carlo particle filtering and smoothing. Adaptive non-linear filtering; ML, MAP, MCMC, and EM estimation of system parameters. Example applications from navigation, remote surveillance, and time series analysis.

    Assessment Methods and Criteria:
    Final exam, home exercises, and project work. The grade of the course is the maximum of the grades of the examination and project work. You need to pass both the examination and the project work to pass the course. To pass the course, you also need to do at least 1/2 of the home exercises. Furthermore if you do (at least) 3/4 of the exercises, your grade increases by one (1 -> 2, 2 -> 3, 3 -> 4, 4 -> 5).


    Study Material:
    Särkkä: Bayesian Filtering and Smoothing (2013) http://users.aalto.fi/~ssarkka/pub/cup_book_online_20131111.pdf, handouts.

    Course Homepage:
    https://mycourses.aalto.fi/course/view.php?id=20984

    Prerequisites:
    Basics of Bayesian inference, multivariate calculus and matrix algebra. Basic knowledge or ability to learn to use Matlab or Octave is needed for completing the exercises. "ELEC-E8740 Basics of sensor fusion" is recommended, and "CS-E5710 Bayesian data analysis" can be useful.

    Grading Scale: 0-5

    Language:
    The course will be taught in English in spring 2019.

  • Schedule

    The exercises and lectures are on Wednesdays in TU1 in Maarintie 8 (TUAS) for the first 2 weeks, and then in T2 (CS-House) on the rest of the weeks. The exercises are at 14:15-15:00 and the lectures are 15:15-17:00. Note that the first lecture is on January 9th and there is no exercise session on that day.

    • 9.1. Overview of Bayesian modeling of time-varying systems (no exercise session before this lecture) --- in TU1
    • 16.1. From linear regression to Kalman filter and beyond-- in TU1
    • 23.1. Bayesian optimal filtering equations and the Kalman filter-- in T2
    • 30.1. Extended Kalman filter and statistical linearization-- in T2
    • 6.2. Unscented Kalman filter, Gaussian Filter, GHKF and CKF-- in T2
    • 13.2. Particle filtering & information on project work-- in T2
    • 20.2. (no lecture, no exercise session)
    • 27.2. Rao-Blackwellized particle filtering-- in T2
    • 27.2. Individual project work starts (= project topic selection deadline)
    • 6.3. Bayesian optimal smoother, Rauch-Tung-Striebel smoothing-- in T2
    • 13.3. Gaussian and particle smoothers-- in T2
    • 20.3. Bayesian estimation of parameters in state space models-- in T2
    • 27.3. Recap of the course topics-- in T2
    • 10.4. Examination-- in place TBA
    • 12.4. Project work deadline


  • Materials

    Course book

    The course book can be found in the following link and is also available, e.g., in Amazon:

    http://users.aalto.fi/~ssarkka/pub/cup_book_online_20131111.pdf

    Additional materials (examples from the book, etc.) can be found on the book's homepage (tab "Resources"):

    http://www.cambridge.org/fi/academic/subjects/statistics-probability/applied-probability-and-stochastic-networks/bayesian-filtering-and-smoothing

    Lecture slides

    After each lecture, the lecture slides will be added below in PDF form.

  • Exercises

    Not available unless: Your User account contains (use: aalto.fi) contains aalto.fi

    There are a total of 10 home work exercise rounds (in addition to the project work described in Section "Project").

    The exercises include pen and paper exercises and coding tasks in Matlab/Octave.  Before each lecture there is an exercise hour (the first week being an exception) which works as follows: In the beginning, you should flag to a list which of the exercises you have completed. One person will be randomly selected to present his/her answer on the drawing board or computer. 

    Note that to pass in the end of the course you must have done and flagged a total of 1/2 of all the exercises during the course. Furthermore if you do (at least) 3/4 of the exercises, your grade increases by one (1 -> 2, 2 -> 3, 3 -> 4, 4 -> 5). The course assistant is also available for help to solving the exercises and demonstrating coding tips and tricks.

  • Project

    Each student should select a project work topic from the list below no later than February 27, 2019 -- we will open a return box below. You can ask for more information about the topics from the course assistant (email: filip.tronarp@aalto.fi). Also note the the last topic is "Your Own Topic". Optimally, the project work would be one where you apply the methods to an application within your own research area.

    Report Submission:
    Please use the project work submission folder in the end of this page.

    Example Topics:
    Below some example topics are listed. Some of them are more of literature based and others more of the hands on type. All topics should still include some example(s) related to simulated or real data. Students are strongly encouraged to come up with a topic of their own.

    1. Find out how the fusion of radar and acceleration sensor measurements works in Apollo Guidance Computer (AGC) and formulate it as a more modern state space model. Simulate and implement the corresponding estimator (EKF).
    2. Simulate the pseudo-range measurements done by GPS receiver and implement EKF or sigma-point filter, which estimates the position of the GPS receiver.
    3. Implement teaching of MLP neural network with EKF, UKF, CKF or other non-linear Kalman filter.
    4. Find out from literature what is a square-root Kalman filter and implement one. Compare the numerical stability of the algorithm to conventional Kalman filter in some almost singular simulated model.
    5. Discretization and Kalman filter based estimation of a physical system, which is modeled as a partial differential equation. For example, a convection-diffusion equation or wave equation.
    6. Phase locked loops (PLL) and their relationship with extended Kalman filter. 
    7. Hidden Markov models (HMM), Viterbi decoder and their relationship with optimal filtering and smoothing.
    8. Restoration of audio signals with EKF or other non-linear filters.
    9. Parameter Estimation in non-linear State Space Models.
    10. Constrained Kalman filtering.
    11. Kalman filtering for linear stochastic differential equations.
    12. Continuous-discrete-time non-linear Kalman filters.
    13. Continuous-time non-linear Kalman filters.
    14. Theory of continuous-discrete time filtering, Fokker-Planck-Kolmogorov equations.
    15. Theory of continuous-time filtering, Zakai equation, Kushner-Stratonovich equation.
    16. The connection between Gaussian process regression and Kalman filtering.
    17. Kalman filters and non-Gaussian measurement noise.
    18. Your Own Topic.

       
    The Report
    The course assignment is returned as a written report in PDF form to the course assistant no later than April 12, 2019. The grading is based on the report. Codes can be included in appendices if they are essential or related to the report. The report should at least contain the following:

    • An Introduction. Explains the research problem in informal terms. Based on this, a fellow student on the course should be able to understand how your project relates to the rest of the course.
    • A Theory section (Materials and Methods). Describes the theory behind the application and/or methodology and cites books and scientific articles, where the theory can be found.
    • Simulation/Results. The method is applied to a simulated or real application. Codes can be included as appendices, if necessary.
    • A Summary (Discussion and Conclusion). Summarizes the results and provides insight into the usability of the method. If applicable, also discusses god/bad sides of the approach.

    Use some word processing software (e.g. [pdf/xe/lua/...]LaTeX, preferably) to typeset your report. You can use some standard article or report template.
     
    Evaluation criteria:
    The course work is a compulsory part of the course and must be passed in order to pass the course. The grades in the higher end of the grading scale will be awarded for outstanding performance and potentially interesting viewpoints into the themes. If you fail this part of the course, you will be given an opportunity to refine your report in order to pass the assignment.
     
    (Frequently) Asked Questions:
    How much time should be allocated to this part of the course?
    Simo has estimated that this part should be a bit more than 1/5 of the whole course. However, as the grade is based on either the exam or the course assignment, those who are aiming for the higher grades should allocate their workload accordingly.

    Language to report in?
    Reports can be written in English and Swedish. However, the easy way out is to write the report in English.




  • Results

    Not available unless: Your User account contains (use: aalto.fi) contains aalto.fi