# Overview
```{image} images/tisp_cover.png
```
These notes aim to provide a solid mathematical background for
a deeper understanding of signal processing problems.
{ref}`ch:set:theory` covers basic definitions of
sets, relations, functions and sequences.
It introduces general Cartesian products and
axiom of choice.
The concept of function is somewhat different
as a function $f: A \to B$ may not be defined
at every point in $A$.
{ref}`ch:bra:chapter` introduces the real line $\RR$
and explains the topology of real line in terms
of neighborhoods, open sets, closed sets, interior,
closure, boundary, accumulation points, covers
and compact sets. Sequences and series of real
numbers are introduced.
The extended real line $\ERL$ is introduced
which plays an important role in optimization problems.
Basic properties of real valued functions $f: X \to \RR$
from an arbitrary set $X$ to real line $\RR$ are introduced.
The graph, epigraph, sublevel and superlevel sets are
defined. Real functions of type $f: \RR \to \RR$ are
discussed and concepts of limits and continuity
are formally defined. Differentiation is introduced
from analytical point of view. Several fundamental
inequalities on real numbers are proved.
{ref}`ch:metric-spaces` covers the topology
of metric spaces, sequences in metric spaces,
functions and continuity, completeness,
compactness. Real valued functions on metric
spaces are discussed in detail covering
concepts of closed functions, semicontinuity,
limit superior and inferior.
{ref}`ch:la:linear-algebra` covers
vector spaces, matrices, linear transformations,
normed linear spaces, inner product spaces,
Banach spaces, Hilbert spaces, eigen value
decomposition, singular values, affine sets
and transformations, and a plethora of
related topics from both algebraic and analytical perspective.
{ref}`ch:cvx:sets:functions` provides an in-depth treatment
of convex sets and functions.
Hyperplanes, halfspaces, general convex sets, lines
and line segments, rays, balls, convex hulls, orthants, simplices,
polyhedra, polytopes, ellipsoids
are discussed.
Convex cones, conic hulls, pointed cones, proper cones, norm cones,
barrier cones are described. Dual cones, polar cones
and normal cones are explained.
Generalized inequalities are introduced.
Convexity preserving operations are described.
Convex function and their properties are covered.
Proofs are provided for the convexity of a large
number of convex functions. Proper convex functions,
indicator functions, sublevel sets, closed and convex
functions, support functions, gauge functions,
quasi convex functions are explained with their
properties and examples.
Topological properties of convex sets including
closure, interior, compactness, relative interior,
line segment property are covered.
Different types of separation theorems are proved.
First order and second order conditions for
convexity of differentiable functions are proved.
Operations which preserve the convexity of functions
are described.
Continuity of convex functions at interior and
boundary points is discussed.
Recession cones and lineality spaces are described.
Directional derivatives for convex functions and
their properties are covered.
Subgradients are introduced for functions which
are convex but not differentiable.
Conjugate functions are developed.
Smoothness and strong convexity of convex functions
is discussed. Infimal convolution is introduced.
{ref}`ch:cvx-opt` develops the theory of
optimization of convex functions under
convex constraints. Topics covered include:
basic definitions of unconstrained and
constrained convex optimization problems,
projections on convex sets, directions of recession,
min common max crossing (MCMC) duality framework,
minimax theorems, saddle point theorem,
stationary points, first order and second order
criteria for optimization of unconstrained functions,
first order criteria for constrained optimization,
descent directions methods, gradient method,
gradient projection method,
Farkas' and Gordan's lemmas, KKT conditions for
minimization of smooth functions under linear
equality and inequality constraints,
feasible directions, tangent cones, optimality
conditions based on tangent cones,
minimization of smooth functions under smooth
inequality and equality constraints, Fritz John
and KKT conditions for the same, Lagrange multipliers,
constraint qualifications, Lagrangian duality,
conjugate duality, Fenchel duality theorem,
linear programming, quadratic programming.