Material
The preliminary versions of the lecture slides can be found below. The slides may still be updated during the course. The lectures are prerecorded and published (at the latest) on Mondays and Wednesdays.
Recommended supplementary reading: J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Springer, 2005 (mainly Chapters 2 and 3), and D. Calvetti and E. Somersalo, Introduction to Bayesian Scientific Computing. Ten Lectures on Subjective Computing, Springer, 2007.
March 1
Practical issues, motivation, compact operators and singular value decomposition, Fredholm equation and its solvability.
Practical issues:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=1326dbef-6bff-45fa-b6fd-acd800f52dec
Well-posed problems, ill-posed problems, (inverse) heat equation example:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a3ca5bbf-e711-498c-8a5a-acd800fb4471
Fredholm equation, compact operators, singular value decomposition, solvability conditions:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=bd592f00-6d47-4420-98b9-acd9007fc65a
Lecture 1
lecture1.pdf222.3KB
March 3
Truncated singular value decomposition, pseudoinverse.
Truncated singular value decomposition solution:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a149aa00-890c-49a6-bb9b-acd900c769e5
Interpretation for matrices and MATLAB:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=8e5cb31a-49cc-4d9d-8774-acd900d34531
Discretization of the heat equation example and solving the corresponding inverse problem using truncated singular value decompositions:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=9c195974-0be0-42b7-bbae-acd900df2ff0
March 8
Morozov discrepancy principle, Tikhonov regularisation and its generalizations.
Review of the truncated SVD solution, Morozov's discrepancy principle and its motivation:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ffa69909-9861-457a-af35-ace100ea5f54
(Basic) Tikhonov regularization, and its motivation, well posedness and theoretical properties:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=69148e33-42a2-4a3d-bcbe-ace100f8c982
Implementing Tikhonov regularization for matrix equations (in MATLAB), application to the (discretized) inverse heat equation:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=1f187112-615a-413f-8024-ace101058837
Generalizations of Tikhonov regularization, i.e. nonlinear inverse problems and more general penalty terms:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=04704008-9b35-43d3-b5d0-ace1010d12db
March 10
Regularization by truncated iterative methods: Landweber-Fridman iteration.
Basic idea of (truncated) iterative methods (for solving inverse problems), Banach fixed point theorem, convergence of Landweber-Fridman iteration:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=f342d724-4941-4571-807d-ace200dd89e7
Regularization properties of Landweber-Fridman iteration, application to the inverse heat equation:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=8556b8c9-3b78-48ac-a3b5-ace7007da16a
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a1094b06-6ee2-4e1e-9ba2-ace700890b66
March 15
Regularization by truncated iterative methods: conjugate gradient method.
Basic idea of Krylov subspace methods, equivalent formulation of a positive definite matrix equation as a quadratic minimization problem:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=93a4edca-86e1-4276-96da-ace900f1bb56
Sequential minimization over lines:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e7e6c002-9ba1-44e0-885d-aceb00a9bfff
Minimization over a hyperplane and its equivalence with sequential minimization over lines for A-conjugate search directions:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=82d4ef39-06d7-4f9d-ae45-aceb00b7ece0
Krylov subspaces and their connection to the construction of A-conjugate search directions in the (preliminary) conjugate gradient algorithm:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=60977dcd-d04c-4691-86ae-aceb00c95bc7
March 17
Conjugate gradient method (cont.), preliminaries of statistical inversion.
Review of the main ideas behind the conjugate gradient method, its standard formulation and two ways of applying it to the inverse heat equation:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e96327ec-1481-4cb3-84c9-acec00eace15
Basic setting on statistical/Bayesian inversion, (very) basics of probability theory (without measure theory):
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=b751d4c0-09b2-4c95-81da-acee007f140c
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a2533392-da32-42fe-b348-acee00c8ba3c
March 22
Preliminaries of statistical inversion (cont.), construction of likelihood.
Poisson process, Gaussian random variables, the central limit theorem (all very informally):
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e02aebbc-ac7c-4e9b-be37-acf3007e34c5
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=326a232f-d682-4a39-8bc5-acf300cedcab
Inverse problems in the Bayesian setting: prior, likelihood and posterior.
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=885dd08a-f47b-4cae-ac44-acf300d59695
Estimators:
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=d64a7ea0-6928-4692-ae31-acf300e19f2a
Construction of likelihood (additive noise model):
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=3a83d9e8-2a44-4470-8a68-acf4006c257e
March 24
Construction of likelihood (cont.), sampling, prior models.
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a707f988-7413-4f57-b107-ab8300b9da62
March 29
Prior models (cont.), n-variate Gaussian densities.
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=91ab0e86-3047-4d5b-b4d8-ab84008885f2
March 31
Improper Gaussian priors, MCMC: Metropolis-Hastings algorithm.
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=fcda6f1d-f094-4294-9fd5-ab8400c44951
April 5
Gibbs sampler, judging the quality of a sample.
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e0aad711-bdad-4e33-a030-ab8800c47b25
April 7
Hyper models.
https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=75430a96-aa39-4f50-b812-ab9100a53ebd