8. Numerical Optimization

A mathematical optimization problem consists of maximizing or minimizing a real valued function under a set of constraints. We shall assume $\mathbb{V}$ to denote a finite dimensional real vector space. Typical examples of $\mathbb{V}$ are ${\mathbb{R}}^n$ and ${\mathbb{S}}^n$.

Formally, we express a mathematical optimization problem as:

$$\begin{array}{r}\begin{array}{rlrlrl}& \text{minimize}& & f_0(\mathbf{x})& & \\ & \text{subject to}& & f_i(\mathbf{x})\le b_i,& & i=1,\dots ,m.\end{array}\end{array}$$

• $\mathbf{x}\in \mathbb{V}$ is the optimization variable of the problem.

• $f_0:\mathbb{V}\to \mathbb{R}$ is the objective function.

• The functions $f_i:\mathbb{V}\to \mathbb{R},i=1,\dots ,m$ are the (inequality) constraint functions.

• The (real scalar) constants $b_1,\dots ,b_m$ are the limits for the inequality constraints.

• A vector $\mathbf{x}\in \mathbb{V}$ is called feasible if it belongs to the domains of $f_0,f_1,\dots ,f_m$ and satisfies all the constraints.

• A vector ${\mathbf{x}}^{\ast }$ is called optimal if is feasible and has the smallest objective value; i.e. for any feasible $\mathbf{z}$, we have $f_0(\mathbf{z})\ge f_0({\mathbf{x}}^{\ast })$.

• An optimal vector is also called a solution to the optimization problem.

• An optimization problem is called infeasible if there is no feasible vector. i.e. there is no vector $\mathbf{x}\in \mathbb{V}$ which satisfies the inequality constraints.

• An infeasible problem doesn’t have a solution.

• A feasible problem may not have a solution if the objective function is unbounded below. i.e. for every feasible $\mathbf{x}$, there exists another feasible $\mathbf{z}$ such that $f_0(\mathbf{z})<f_0(\mathbf{x})$.

• If a feasible problem is not unbounded below, then it may have one or more solutions.