Schedule: 11.09.2019 - 04.12.2019
Contact information for the course (applies in this implementation):
Neda B. Marvasti
Teaching Period (valid 01.08.2018-31.07.2020):
Content (valid 01.08.2018-31.07.2020):
Postgraduate level knowledge from one of the fields of computer and information science. The actual contents of the course vary from year to year. The course can be lectured, or arranged in seminar form.
Details on the course content (applies in this implementation):
CS-E4070 - Approximate Bayesian Computation: 2019 (3 ECTS)
Responsible Teacher: Neda B. Marvasti
Level of the Course: Master's and PhD level
Teaching Period: I-II (Autumn)
Description: Approximate Bayesian Computation (a.k.a. ABC, likelihood-free inference) is a new class of computational inference methods that can be used when the likelihood function is difficult to evaluate or unknown, and one has a simulator for generating data that (hopefully) resemble observations when generated with correct parameters. The underlying intuition is that similar model parameters are likely to generate similar data, but the practice is of course a bit more complex...
ABC has applications from medicine to particle physics, and is expected to revolutionise computational sciences that cannot apply traditional statistical methods.
Assessment Methods and Criteria (valid 01.08.2018-31.07.2020):
To be specified at the start of the course.
Elaboration of the evaluation criteria and methods, and acquainting students with the evaluation (applies in this implementation):
To pass the course you have to :
- Give a presentation (approx 30 min) on a related topic (50%)
- Act as an opponent in a presentations (5%)
- Finish an assignment (30%)
- Peer-review 2 assignments (5%)
- Present your assignment (approx 5-10 min) (10%)
First meeting on 11 September at 12:15 in room T3. An introduction to ABC will be given along with agreeing about seminar duties and schedule.
Details on calculating the workload (applies in this implementation):
Opponents should have a look at the topic beforehand and actively ask questions. It's ok to substitute missed duties at other times as well.
Lectures should cover the general idea of the algorithm, theory and examples, and last about 30 minutes. Consider implementing the algorithm yourself and presenting how it works (this is probably the best way to learn!).
The assignment is to perform a case study on some proper problem, either with real data or a toy model, and produce a "notebook" (Jupyter Notebook) with introduction, methods, codes and results. Please check your topic with lecturer.
The due date for assignments is 27 November, after which there's one week for peer-reviewing. The assignments can be done within the ELFI framework.
Study Material (valid 01.08.2018-31.07.2020):
Usually some new study book or collection of articles.
Details on the course materials (applies in this implementation):
Check introduction.pdf in materials
Some material in arbitrary order (also check references therein):
Review papers to find references of your selected topic (do not present the review papers itself):
- Marin, J.-M., Pudlo, P., Robert, C. P., and Ryder, R. J. (2012). Approximate Bayesian computational methods. Statistics and Computing, 22(6):1167–1180
- Karabatsos, George, and Fabrizio Leisen. "An approximate likelihood perspective on ABC methods." Statistics Surveys 12 (2018): 66-104.
- Lintusaari, J., Gutmann, M. U., Dutta, R., Kaski, S., and Corander, J. (2016). Fundamentals and recent developments in approximate Bayesian computation, Systematic Biology, doi: 10.1093/sysbio/syw077
- Drovandi, C. C. and A. N. Pettitt (2011). Estimation of parameters for macroparasite population evolution using approximate Bayesian computation. Biometrics 67, 225– 233.
- ABC-MCMC: Markov chain Monte Carlo without likelihoods
- ABC-SMC: S. A. Sisson, Y. Fan and Mark M. Tanaka. Sequential Monte Carlo without likelihoods, PNAS 104, 6, 2007
- ABC-PMC: Beaumont, M. A., C. P. Robert, J.-M. Marin, and J. M. Corunet (2009). Adaptivity for abc algorithms: The ABC-PMC scheme.
- Stuart Barber, Jochen Voss and Mark Webster. The rate of convergence for approximate Bayesian computation, Electronic Journal of Statistics, 9, 2015
- Dennis Prangle, Adapting the ABC Distance Function.
- Approximate sufficiency : Approximately sufficient statistics and
- Entropy/loss minimisation: On optimal selection of summary
statistics for approximate Bayesian computation.
- Mutual information: Considerate approaches to constructing summary statistics for ABC model selection
- Regularisation approaches: Choosing the summary statistics and the acceptance rate in approximate Bayesian computation
- Likelihood distance: Approximate Bayesian computation with
indirect summary statistics.
- Partial least squares: Efficient approximate
Bayesian computation coupled with Markov chain Monte Carlo without likelihood
- Linear regression: Constructing summary statistics for approximate Bayesian computation: Semi-automatic ABC.
- Boosting: A novel approach
for choosing summary statistics in approximate Bayesian computation.
- Auxiliary likelihood methods: Approximate Bayesian computation with indirect summary statistics.
- Neural networks: Non-linear regression models for Approximate Bayesian Computation
Check slide 15 of the introduction slide
Useful for lecture 6:
Check slide 16 in the introduction slide
Useful for lecture 7:
- Michael U. Gutmann and Jukka Corander (2016). Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models, Journal of Machine Learning Research 17, 1-47
- Alexander Moreno, Tameem Adel, Edward Meeds, James M. Rehg, Max Welling. Automatic Variational ABC, ArXiv preprint
- Minh-Ngoc Tran, David J. Nott, Robert Kohn. Variational Bayes with Intractable Likelihood, ArXiv preprint
- Mescheder, Lars, Sebastian Nowozin, and Andreas Geiger. "Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks." Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
- Michael U. Gutmann, Ritabrata Dutta, Samuel Kaski and Jukka Corander. Statistical Inference of Intractable Generative Models via Classification, ArXiv preprint
- Ritabrata Dutta, Jukka Corander, Samuel Kaski, Michael U. Gutmann. Likelihood-free inference by ratio estimation (LFIRE), ArXiv preprint
- Ong VM, Nott DJ, Tran MN, Sisson SA, Drovandi CC. Likelihood-free inference in high dimensions with synthetic likelihood. Computational Statistics & Data Analysis. 2018 Dec 1;128:271-91.
Substitutes for Courses (valid 01.08.2018-31.07.2020):
T-61.6010 Special Course in Computer and Information Science I, T-61.6020 Special Course in Computer and Information Science II, T-61.6030 Special Course in Computer and Information Science III, T-61.6040 Special Course in Computer and Information Science IV, T-61.6050 Special Course in Computer and Information Science V, T-61.6060 Special Course in Computer and Information Science IV, CS-E4010 Special Course in Machine Learning and Data Science I, CS-E4020 Special Course in Machine Learning and Data Science II, CS-E4030 Special Course in Machine Learning and Data Science III, CS-E4040 Special Course in Machine Learning and Data Science IV, CS-E4050 Special Course in Machine Learning and Data Science V, CS-E4060 Special Course in Machine Learning and Data Science VI.
Grading Scale (valid 01.08.2018-31.07.2020):
0-5, may be graded with pass/fail
Further Information (valid 01.08.2018-31.07.2020):
The contents of the course vary.
Additional information for the course (applies in this implementation):
A maximum of 30 students will be accepted to the course.
Details on the schedule (applies in this implementation):
Meeting dates (subject to changes!):
- 11 September (Introduction & practicals)
- 18 September (Rejection sampling +MCMC-ABC +SMC-ABC), (Convergence, validation, post-processing in ABC), (Distance metrics & summary statistics)
- 25 September(Summary statistics selection methods)
- 2 October (Summary statistics selection methods)
- 9 October (Sampling-based ABC methods)
- 16 October (Sampling-based ABC methods)
- 30 October (Regresion based ABC methods)
- 6 November (BOLFI, Variational methods in ABC)
- 13 November (GANs for LIFE)
- 20 November (High-dimensional ABC)(Model selection in ABC)
- 27 November (Assignment presentations)
- Teacher: Neda Barzegar Marvasti