Lecture 6 - Dynamic Programming, discrete problem, B1.2-1.3

Introduction to the course, motivation and organization

Classification of problems. Topics covered on the course:

  • calculus of variations
  • optimal control problem
  • dynamic programming

Dynamic Programming problem and terminology, discrete vs. continuous time, finite vs. infinite state, finite vs. infinite horizon, state, control, disturbance, deterministic vs. stochastic systems, control law and policy, admissible control laws, reachable states, open vs. closed loop controls, cost-to-go modeling, recursive computation, multistage decision making

Modeling dynamic optimization problems

Dynamic Programming algorithm

Sections from the book: Bertsekas 1.2, 1.3