Lecture 4 - Continuous time problem, HJB equation, B3.1-3.2,K3.11

Continuous time problem: formulation and discretization

Hamilton-Jacobi-Bellman (HJB) equations, derived from the corresponding discrete problem and using dynamic programming, partial differential equation (partial derivates with respect to time t and space x), cost to go in continuous time J(t,x), Hamiltonian H=g+p'f, p is costate variable (Lagrange "multiplier"), sufficient conditions

first solution to the control problem, we will derive them in other form later using calculus of variations

Sections from the book: Bertsekas 3.1-3.2, Kirk 3.11