Topic: Lectures and quizzes | MS-C2111 - Stochastic Processes, Lecture, 1.11.2021-15.12.2021

Topic outline

Lectures and quizzes
Lectures are delivered online via Zoom (link: https://aalto.zoom.us/j/66263981915). Video clips of lectures appear here afterwards (unless something fails). In addition to watching the lectures, you are recommended to study the lecture notes which provide more in-depth information.
Before each lecture you are advised to do a Preparatory quiz (see below), from which you get also points. Note that each quiz has deadline when the lecture starts (you cannot submit solutions later).
- Select activity 1 Markov chains and stochastic modelsMarkov chain,...
  
  1 Markov chains and stochastic models
  Markov chain, transition matrix, transition diagram, applications.
  Recordings: Motivation (13 min), Markov processes (25 min), Transition matrix (25 min), Path probabilities (4 min), Simulation (8 min)
- Select activity Preparatory quiz 1
  
  Preparatory quiz 1
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 1 slides
  
  Not available unless: You are a(n) Student
  
  Lecture 1 slides File PDF
- Select activity Lecture 1 example code
  
  Lecture 1 example code Folder
- Select activity 2 Markov chains in the long runInvariant distribut...
  
  2 Markov chains in the long run
  Invariant distributions, limiting distributions, connectivity, periodicity.
  Recordings: Invariant and limit distributions + Brand loyalty (29 min), Examples 2 (Ehrenfest) and 3 (23 min), Connectivity = irreducibility (21 min), Periodicity + convergence (11 min)
  Pointer to Levin-Peres-Wilmer online book.
- Select activity Preparatory quiz 2
  
  Preparatory quiz 2
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 2 slides
  
  Not available unless: You are a(n) Student
  
  Lecture 2 slides File PDF
- Select activity Lecture 2 example code
  
  Lecture 2 example code Folder
- Select activity 3 Markov additive processesMarkov additive process...
  
  3 Markov additive processes
  Markov additive process, empirical frequency, ergodicity, convergence of time averages
  Recordings: What happens if reducible or periodic (19 min); Examples (coins and Katiskakauppa) (28 min); Additive Markov process & Finite time horizon (40 min); Ergodicity (5 min)
- Select activity Preparatory quiz 3
  
  Preparatory quiz 3
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 3 slides
  
  Not available unless: You are a(n) Student
  
  Lecture 3 slides File PDF
- Select activity 4 Passage times and hitting probabilitiesExpected ...
  
  4 Passage times and hitting probabilities
  Expected passage time, hitting probability, gambler's ruin.
  Recordings: part 1 (46 min), part 2 (43 min).
- Select activity Preparatory quiz 4
  
  Preparatory quiz 4
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 4 slides
  
  Lecture 4 slides File PDF
- Select activity Lecture 4 example code
  
  Lecture 4 example code Folder
- Select activity 5 General Markov chains and random walksInfinite t...
  
  5 General Markov chains and random walks
  Infinite transition matrix, convergence, reversible Markov chain, random walk
  Recordings: part 1 (45 min), part 2 (48 min).
- Select activity Preparatory quiz 5
  
  Preparatory quiz 5
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 5 slides
  
  Lecture 5 slides File PDF
- Select activity Lecture 5 codes
  
  Lecture 5 codes Folder
- Select activity 6 Branching processesGenerating function, expected...
  
  6 Branching processes
  Generating function, expected population size, extinction probability
  Recordings: part 1 (44 min), part 2 (47 min)
- Select activity Preparatory quiz 6
  
  Preparatory quiz 6
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 6 slides
  
  Lecture 6 slides File PDF
- Select activity Lecture 6 code
  
  Lecture 6 code File TXT
- Select activity 7 Random point patternsRandom point pattern, indep...
  
  7 Random point patterns
  Random point pattern, independent scattering, exponential distribution
  Recordings: part 1 (45 min), part 2 (47 min)
- Select activity Preparatory quiz 7
  
  Preparatory quiz 7
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 7 slides
  
  Lecture 7 slides File PDF
- Select activity Lecture 7 code
  
  Lecture 7 code Folder
- Select activity 8 Poisson processes and renewal processesRecording...
  
  8 Poisson processes and renewal processes
  
  Recordings:
  
  Superposed Poisson (43 min), Compound Poisson (19 min), Thinned Poisson (18 min), Renewal process (12 min)
- Select activity Preparatory quiz 8
  
  Preparatory quiz 8
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 8 slides
  
  Lecture 8 slides File PDF
- Select activity Lecture 8 code
  
  Lecture 8 code File M
- Select activity 9 Continuous-time Markov chains in finite time hor...
  
  9 Continuous-time Markov chains in finite time horizon
  Transition semigroup, generator matrix, Poisson modulated chains
  Recordings: part 1 (46 min), part 2 (49 min)
- Select activity Preparatory quiz 9
  
  Preparatory quiz 9
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 9 slides
  
  Lecture 9 slides File PDF
- Select activity Lecture 9 code
  
  Lecture 9 code File M
- Select activity 10 Analysis of continuous-time Markov chainsJump r...
  
  10 Analysis of continuous-time Markov chains
  Jump rate and jump probability, overclocking, determining the generator in applications
  Recordings: part 1 (47 min), part 2 (52 min, includes a question+discussion at the end)
- Select activity Preparatory quiz 10
  
  Preparatory quiz 10
  
  Students must
  
  View
  
  Receive a grade
- Select activity Lecture 10 slides
  
  Lecture 10 slides File PDF
- Select activity Lecture 10 code
  
  Lecture 10 code Folder
- Select activity 11 Martingales and information processesMartingale...
  11 Martingales and information processes
  Martingale, submartingale, supermartingale. Conditional expectation with respect to information. Martingale transforms of Markov chains.
  Note: The online lecture 11 is cancelled due to personal reasons. Please watch the following recordings from last year instead (total 85 minutes).
  
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (6 min)
  Martingales are used as stochastic models, and also as theoretical tools for analysing other stochastic processes.
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (12 min)
  Relative conditional expectation is defined as a random variable representing an outsider's view of an expected value computed by an insider.
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (11 min)
  Expected value of the number of aces in the hand of player 2, from the point of view of player 1, player 2, and the dealer.
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (2 min)
  Functional dependence is a special form of dependence, not to be confused with stochastic dependence
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (17 min)
  Arithmetic rules for working with conditional expectations
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (6 min)
  Definition: Martingale, submartingale, supermartingale
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (7 min)
  Random walks with zero drift are martingales
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (11 min)
  Prediction martingales form an important class of stochastic processes having convergent paths
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (4 min)
  Nonnegative martingales and bounded martingales converge pathwise with probability one
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (9 min)
  Functional transforms of Markov chains with zero drift are martingales, and so are normalised branching processes.
- Select activity Preparatory quiz 11
  
  Preparatory quiz 11
  
  Students must
  
  View
  
  Receive a grade
- Select activity 12 Stopped martingales and optional timesOptional ...
  12 Stopped martingales and optional times
  Optional time, previsible process, stopped martingale, optional stopping theorem
  Online lecture cancelled. Old video recordings are here (total 92 min)
  
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (11 min)
  Doubling is a famous gambling strategy apparently providing a risk-free profit
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (17 min)
  Net profit from adaptive betting can be represented as an integral process against a unit return process
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (14 min)
  Integrating a previsible process against a martingale produces a martingale
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (18 min)
  Stopping a martingale at an optional time produces a martingale
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (18 min)
  Doob's optional stopping theorem returns the expected value of a martingale observed at an optional time.
  Video Player is loading.
  Current Time 0:00
  Duration -:-
  Loaded: 0%
  Stream Type LIVE
  Remaining Time -:-
  
  1x
  2x
  1.75x
  1.5x
  1.25x
  1x, selected
  0.75x
  0.5x
  Chapters
  descriptions off, selected
  captions settings, opens captions settings dialog
  captions off, selected
  This is a modal window.
  The media could not be loaded, either because the server or network failed or because the format is not supported.
  Beginning of dialog window. Escape will cancel and close the window.
  TextColorOpacity
  Text BackgroundColorOpacity
  Caption Area BackgroundColorOpacity
  Font Size
  Text Edge Style
  Font Family
  End of dialog window.
  (14 min)
  Summary of main learning outcomes and closing the course.
- Select activity Preparatory quiz 12
  
  Preparatory quiz 12
  
  Students must
  
  View
  
  Receive a grade

MS-C2111 - Stochastic Processes, Lecture, 1.11.2021-15.12.2021

Topic outline

Lectures and quizzes

1 Markov chains and stochastic models

2 Markov chains in the long run

3 Markov additive processes

4 Passage times and hitting probabilities

5 General Markov chains and random walks

6 Branching processes

7 Random point patterns

8 Poisson processes and renewal processes

9 Continuous-time Markov chains in finite time horizon

10 Analysis of continuous-time Markov chains

11 Martingales and information processes

12 Stopped martingales and optional times

Students

Teachers

About service