Lecture 11 - Summary

Two kinds of problems:

  • Discrete time problems
  • Continuous time problems

Two approaches:

  • Dynamic Programming
  • Calculus of variations

HJB equation is DP algorithm in continuous time problems

Bellman equation is DP algorithm for infinite horizon discounted, stationary problem

Calculus of variations was used in deriving necessary conditions for the control problem

Stochastics was only considered in discrete time problems