MS-E1654 - Computational Inverse Problems D, Lecture, 27.2.2023-14.4.2023

Topic outline

• Materials
  The preliminary versions of the lecture slides can be found below. The slides may still be updated during the course. The lectures have been prerecorded and published below; some of the practical information in the lecture recordings may be outdated.

  Recommended supplementary reading: J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Springer, 2005 (mainly Chapters 2 and 3), and D. Calvetti and E. Somersalo, Introduction to Bayesian Scientific Computing. Ten Lectures on Subjective Computing, Springer, 2007.

  • Select activity Lecture 1

    Week 1/1

    Practical issues, motivation, compact operators and singular value decomposition, Fredholm equation and its solvability.

    Well-posed problems, ill-posed problems, (inverse) heat equation example:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a3ca5bbf-e711-498c-8a5a-acd800fb4471
    

    Fredholm equation, compact operators, singular value decomposition, solvability conditions:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=bd592f00-6d47-4420-98b9-acd9007fc65a

    
    

    Download folder

    • Lecture 1
      • lecture1.pdf
  • Select activity Lecture 2

    Week 1/2

    Truncated singular value decomposition, pseudoinverse.

    Truncated singular value decomposition solution:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a149aa00-890c-49a6-bb9b-acd900c769e5
    

    Interpretation for matrices and MATLAB:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=8e5cb31a-49cc-4d9d-8774-acd900d34531
    

    Discretization of the heat equation example and solving the corresponding inverse problem using truncated singular value decompositions:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=9c195974-0be0-42b7-bbae-acd900df2ff0
    

    Download folder

    • Lecture 2
      • lecture2.pdf
  • Select activity Lecture 3

    Week 2/1

    Morozov discrepancy principle, Tikhonov regularisation and its generalizations.

    Review of the truncated SVD solution, Morozov's discrepancy principle and its motivation:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ffa69909-9861-457a-af35-ace100ea5f54

    (Basic) Tikhonov regularization, and its motivation, well posedness and theoretical properties:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=69148e33-42a2-4a3d-bcbe-ace100f8c982
    

    Implementing Tikhonov regularization for matrix equations (in MATLAB), application to the (discretized) inverse heat equation:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=1f187112-615a-413f-8024-ace101058837
    

    Generalizations of Tikhonov regularization, i.e. nonlinear inverse problems and more general penalty terms:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=04704008-9b35-43d3-b5d0-ace1010d12db
    

    
    

    Download folder

    • Lecture 3
      • lecture3.pdf
  • Select activity Lecture 4

    Week 2/2

    Regularization by truncated iterative methods: Landweber-Fridman iteration.

    Basic idea of (truncated) iterative methods (for solving inverse problems), Banach fixed point theorem, convergence of Landweber-Fridman iteration:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=f342d724-4941-4571-807d-ace200dd89e7
    

    Regularization properties of Landweber-Fridman iteration, application to the inverse heat equation:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=8556b8c9-3b78-48ac-a3b5-ace7007da16a
    

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a1094b06-6ee2-4e1e-9ba2-ace700890b66
    

    Download folder

    • Lecture 4
      • lecture4.pdf
  • Select activity Lecture 5

    Week 3/1

    Regularization by truncated iterative methods: conjugate gradient method.
    

    Basic idea of Krylov subspace methods, equivalent formulation of a positive definite matrix equation as a quadratic minimization problem:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=93a4edca-86e1-4276-96da-ace900f1bb56

    Sequential minimization over lines:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e7e6c002-9ba1-44e0-885d-aceb00a9bfff
    

    Minimization over a hyperplane and its equivalence with sequential minimization over lines for A-conjugate search directions:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=82d4ef39-06d7-4f9d-ae45-aceb00b7ece0
    

    Krylov subspaces and their connection to the construction of A-conjugate search directions in the (preliminary) conjugate gradient algorithm:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=60977dcd-d04c-4691-86ae-aceb00c95bc7

    
    

    Download folder

    • Lecture 5
      • lecture5.pdf
  • Select activity Lecture 6

    Week 3/2

    Conjugate gradient method (cont.), preliminaries of statistical inversion.
    

    Review of the main ideas behind the conjugate gradient method, its standard formulation and two ways of applying it to the inverse heat equation:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e96327ec-1481-4cb3-84c9-acec00eace15
    
    

    Basic setting on statistical/Bayesian inversion, (very) basics of probability  theory (without measure theory):

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=b751d4c0-09b2-4c95-81da-acee007f140c
    

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a2533392-da32-42fe-b348-acee00c8ba3c
    

    Download folder

    • Lecture 6
      • lecture6.pdf
  • Select activity Lecture 7

    Week 4/1

    Preliminaries of statistical inversion (cont.), construction of likelihood.
    

    Poisson process, Gaussian random variables, the central limit theorem (all very informally):

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e02aebbc-ac7c-4e9b-be37-acf3007e34c5
    

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=326a232f-d682-4a39-8bc5-acf300cedcab
    

    Inverse problems in the Bayesian setting: prior, likelihood and posterior.

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=885dd08a-f47b-4cae-ac44-acf300d59695
    

    Estimators:

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=d64a7ea0-6928-4692-ae31-acf300e19f2a
    

    Construction of likelihood (additive noise model):

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=3a83d9e8-2a44-4470-8a68-acf4006c257e
    

    Download folder

    • Lecture 7
      • lecture7.pdf
  • Select activity Lecture 8

    Week 4/2

    Construction of likelihood (cont.), sampling, prior models.
    

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a707f988-7413-4f57-b107-ab8300b9da62
    
    

    Download folder

    • Lecture 8
      • lecture8.pdf
  • Select activity Lecture 9

    Week 5/1

    Prior models (cont.), n-variate Gaussian densities.

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=91ab0e86-3047-4d5b-b4d8-ab84008885f2
    

    Download folder

    • Lecture 9
      • lecture9.pdf
  • Select activity Lecture 10

    Week 5/2

    Improper Gaussian priors, MCMC: Metropolis-Hastings algorithm.

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=fcda6f1d-f094-4294-9fd5-ab8400c44951
    

    Download folder

    • Lecture 10
      • lecture10.pdf
  • Select activity Lecture 11

    Week 6/1

    Gibbs sampler, judging the quality of a sample.

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e0aad711-bdad-4e33-a030-ab8800c47b25
    

    Download folder

    • Lecture 11
      • lecture11.pdf
  • Select activity Lecture 12

    Week 6/2

    Hyper models.

    https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=75430a96-aa39-4f50-b812-ab9100a53ebd
    

    Download folder

    • Lecture 12
      • lecture12.pdf