ELEC-E5424 - Convex optimization D, Lecture, 6.9.2023-14.12.2023

Topic outline

• Select topic General

  General
  Schedule

  	Lectures (Wed. 9.15-11:00)		Exercises (Thu. 12:15-14:00)
  13.9	Lect. 1: T3 Computer Science build.	14.9	Lect. 2/Exer.: TU7 Maarintie 8
  20.9	Lect. 3: T3 Computer Science build.	21.9	Lect. 4/Exer.: TU7 Maarintie 8
  27.9	Lect. 5: T3 Computer Science build.	28.9	Lect. 6/Exer.: TU7 Maarintie 8
  4.10	Lect. 7: T3 Computer Science build.
  11.10	Lect. 8: T3 Computer Science build.	12.10	Exer. 1: TU7 Maarintie 8
  	Exam week (no teaching)		Exam week (no teaching)
  25.10	Lect. 9: T3 Computer Science build.	26.10	Lect. 10/Exer.: TU7 Maarintie 8
  1.11	Lect. 11: T3 Computer Science buil.	2.11	Exer. 2: TU7 Maarintie 8
  8.11	Lect. 12: T3 Computer Science buil.
  15.11	Lect. 13: T3 Computer Science buil.	16.11	Exer. 3: TU7 Maarintie 8
  22.11	Lect. 14: T3 Computer Science buil.
  29.11	Exer. 4: T3 Computer Science build.
  	Preparation week		Preparation week
  13.12	Exam (9:00-11:00): T3

  
  

  Textbook and Optional References

  •        Stephen Boyd; Lieven Vandenberghe, Convex Optimization (Textbook!)

  •        Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications

  •        Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course

  •        J. Gallier and J. Quaintance, Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Engineering

  Course Requirements and Grading

  Requirements:

  •        4 homework assignments. Homeworks will normally be assigned on Wendsdays and will be due in 2 weeks.

  •        Final Exam. The format will be decided depending on the sitation.

  Grading: 

  Homeworks: 60%. Exam: 40%. These weights are approximate. We reserve the right to change them later. Can be also discussed.

  Description

  Concentrates on recognizing and solving (using standard packages) convex optimization problems that arise in practice!

  •        Convex sets, functions, and optimization problems.

  •        Least-squares, linear and quadratic programs.

  •        Semidefinite programming (SDP).

  •        Minimax, extremal volume, and other problems with geometric interpretation.

  •        Optimality conditions, duality theory, theorem of alternatives.

  •        Introduction to unconstrained optimization algorithms.

  •        Introduction to interior-point methods for constrained optimization.

  •        Applications.

  Course Objectives

  •        to give the tools and training to recognize convex optimization problems that arise in electrical engineering and computer science

  •        to present the basic theory of such problems, concentrating on results that are useful in computations

  •        to give a thorough understanding of how such problems are solved, and some experience in solving them

  •        to give the background required to use the standard methods and software packages (CVX toolbox) in your own research work

  •        to give a number of examples of successful application of convex optimization techniques for solving problem in applied mathematics, computer science, statistics, electrical engineering, and science and engineering in general

  
  

  • Select activity Announcements

    

    Announcements Forum
  • Select activity Introduction and Overview

    

    Introduction and Overview File PDF

    What the course is about

  • Select activity Useful Concepts and Facts from Linear Algebra

    

    Useful Concepts and Facts from Linear Algebra File PDF

    Some concepts from linear algebra that are heavily used in the textbook for the course and in general in convex optimization.

  • Select activity Convex Sets

    

    Convex Sets File PDF

    subspace, afiine set, convex set, convex cone (simple examples and properties)

    combination and hulls
    

    ellipsoids, polyhedra, norm balls
    

    affine and projective transformations
    

    separating hyperplanes
    

    generalized inequalities
    

  • Select activity Convex Functions

    

    Convex Functions File PDF

    convex functions, epigraph (simple examples, elementary properties)

    more examples, more properties
    

    Jensen's inequality
    

    quasiconvex, quasiconcave functions
    

    log-convex and log-concave functions
    

    K-convexity
    

  • Select activity Convex Optimization Problems

    

    Convex Optimization Problems File PDF

    abstract form problem

    standard form problem

    convex optimization problem
    

    standard form with generalized inequalities
    

    mulitcriterion optimization
    

    restriction and relaxation
    

  • Select activity Linear and Quadratic Problems, Semidefinite Programming (SDP) Format

    

    Linear and Quadratic Problems, Semidefinite Programming (SDP) Format File PDF

    linear programming (examples and applications)

    linear fractional programming 

    quadratic optimization problems 

    (quadratically constrained) quadratic programming 

    examples and applications 

    Semidefinite programming 

    applications

  • Select activity CVX Tutorial

    

    CVX Tutorial File PDF

    How to use solvers to solve convex optimization problems

  • Select activity Duality

    

    Duality File PDF

    Lagrange dual function

    Lagrange dual problem

    strong duality and Slater's condition

    KKT optimality conditions

    sensitivity analysis

    duality in generalized inequalities notation

    theorem of alternatives

  • Select activity Smooth Unconstrained Minimization Algorithms

    

    Smooth Unconstrained Minimization Algorithms File PDF

    terminology

    general descent method

    line search types

    gradient method

    steepest descent method

    Newton's method

  • Select activity Introduction to Sequential Unconstrained Minimization for Constrained Optimization

    

    Introduction to Sequential Unconstrained Minimization for Constrained Optimization File PDF

    brief history of SUMT & IP methods

    logarithmic barrier function

    central path

    basic SUMT

  • Select activity Assignment 1

    Not available unless: You are a(n) Student

    

    Assignment 1

    Convex Sets

    Due to October 8, 2023

  • Select activity Assignment 2

    Not available unless: You are a(n) Student

    

    Assignment 2

    Convex Functions

    Due to October 29, 2023.

  • Select activity Solutions to Assignment 2

    Not available unless: You are a(n) Student

    

    Solutions to Assignment 2 File PDF
  • Select activity Assignment 3

    Not available unless: You are a(n) Student

    

    Assignment 3

    Convex Problems

    Due to November 12, 2023.

  • Select activity Assignment 4

    Not available unless: You are a(n) Student

    

    Assignment 4

    Duality, Applications, and Algorithms

    Last Assignment!

    Due to November 26, 2023.