The main topics of the course are:
- Solving discrete problems using DP-algorithm (including problems having stochastics and imperfect state information). Bertsekas Sections 1-5. Lectures 1-3.
- Solving continuous problems using calculus of variations (including different end point and corner conditions). Kirk Section 4. Lectures 4-6.
- Solving optimal control problems (including different end point conditions, minimum principle, singular intervals). Kirk Section 5. Lectures 7-9
- Solving optimal control problems using HJB equations. Kirk Section 3.11. Lecture 4
- Solving infinite horizon discounted problems (including Bellman equation, value and policy iterations). Bertsekas vol 2. Section 1. Lecture 10.
- Learning the relevant concepts and terminology.
Note that the main equations are given in the exam paper and you don't have to learn them by heart. Also, bringing a calculator (memory emptied) to the exam is not a bad idea.
Suitable problems can be found in the exercises. You can practice solving them yourself. Similar problems and their solutions are presented in the course books. You can also find many solved problems in the Internet.