4.6. Dual Spaces

Recall from Definition 4.57 that $\mathcal{L}(\mathbb{V},\mathbb{W})$ denotes the vector space of linear transformations from $\mathbb{V}$ to $\mathbb{W}$ where both $\mathbb{V}$ and $\mathbb{W}$ are vector spaces. In Example 4.3, we showed that $\mathbb{F}$ is a vector space in its own right.

In this section, we explore the vector space of linear maps(transformations, functions, operators) from a vector space $\mathbb{V}$ to its field of scalars $\mathbb{F}$; i.e., $\mathbb{W}=\mathbb{F}$.

4.6.1. Linear Functionals

Definition 4.83 (Linear functional)

A linear functional on $\mathbb{V}$ is a linear map $\mathbf{f}:\mathbb{V}\to \mathbb{F}$ from a vector space $\mathbb{V}$ to its field of scalars $\mathbb{F}$. A linear functional is a member of the vector space $\mathcal{L}(\mathbb{V},\mathbb{F})$.

A linear functional is both homogeneous and additive. If $\mathbf{f}$ is a linear functional, then

$$\mathbf{f}(t\mathbf{x}+\mathbf{y})=t\mathbf{f}(\mathbf{x})+\mathbf{f}(\mathbf{y})$$

holds true for all $\mathbf{x},\mathbf{y}\in \mathbb{V}$.

Recall that two functions are considered equal if they have identical domain and they produce same output for every value in the domain.

Definition 4.84 (Equality of linear functionals)

Two linear functionals $\mathbf{f}$ and $\mathbf{g}$ in ${\mathbb{V}}^{\ast }$ are considered equal if

$$\mathbf{f}(\mathbf{v})=\mathbf{g}(\mathbf{v})\mathrm{\forall }\mathbf{v}\in \mathbb{V}$$

Lemma 4.43 (Equality of linear functionals on a basis)

Let $\mathbb{V}$ be a finite dimensional vector space and $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be a basis of $\mathbb{V}$.

If two linear functionals agree on the basis vectors, then they are equal.

Proof. Let $\mathbf{f},\mathbf{g}\in {\mathbb{V}}^{\ast }$. Any $\mathbf{v}\in \mathbb{V}$ can be written as:

$$\mathbf{v}=\sum _{i=1}^n{c}_i{\mathbf{v}}_i.$$

Due to linearity,

$$\mathbf{f}(\mathbf{v})=\sum _{i=1}^n{c}_i\mathbf{f}({\mathbf{v}}_i)\text{ and }\mathbf{g}(\mathbf{v})=\sum _{i=1}^n{c}_i\mathbf{g}({\mathbf{v}}_i).$$

Thus, if $\mathbf{f}({\mathbf{v}}_i)=\mathbf{g}({\mathbf{v}}_i)$ for $i=1,\dots ,n$, then $\mathbf{f}$ and $\mathbf{g}$ equal on every $\mathbf{v}\in \mathbb{V}$.

4.6.2. Dual Spaces

Definition 4.85 (Dual space)

The dual space of a vector space $\mathbb{V}$, denoted by ${\mathbb{V}}^{\ast }$, is the set of all of the linear functionals on $\mathbb{V}$.

In other words: ${\mathbb{V}}^{\ast }=\mathcal{L}(\mathbb{V},\mathbb{F})$.

Observation 4.4

Since the space of linear transformations from $\mathbb{V}$ to $\mathbb{F}$ (as a vector space) is a vector space, hence ${\mathbb{V}}^{\ast }$ is a vector space. The elements of the dual space are the linear functionals.

Every vector space has a zero vector. The dual space must have one too.

Definition 4.86 (Zero functional)

A zero functional $\mathbf{0}:\mathbb{V}\to \mathbb{F}$ is a linear functional which maps every vector $\mathbf{v}\in \mathbb{V}$ to $0\in \mathbb{F}$.

$$\mathbf{0}(\mathbf{v})=0\mathrm{\forall }\mathbf{v}\in \mathbb{V}.$$

It is easy to see that the zero functional is linear and indeed a member of $\mathcal{L}(\mathbb{V},\mathbb{F})$.

We will be using the same symbol $\mathbf{0}$ to denote both the zero functional in ${\mathbb{V}}^{\ast }$ and the zero vector in $\mathbb{V}$. It should be clear from the context which one is being referred to.

If $\mathbf{f}\ne \mathbf{0}$, then there exists at least one $\mathbf{x}\in \mathbb{V}$ such that $\mathbf{f}(\mathbf{x})\ne 0$.

When we have a vector space, it is natural to ask for its dimension and provide a way to build a basis for the space.

4.6.3. Basis for Dual Space

Theorem 4.95 (Basis for dual space)

Let $\mathbb{V}$ be a finite dimensional vector space. Let $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be a basis of $\mathbb{V}$.

For each $i=1,\dots ,n$, define a linear functional ${\mathbf{f}}_i:\mathbb{V}\to \mathbb{F}$ by setting:

$$\begin{array}{r}{\mathbf{f}}_i({\mathbf{v}}_j)=\{\begin{array}{lllll}1& & \text{ if }& & i=j\\ 0& & \text{ if }& & i\ne j\end{array}\end{array}$$

and then extending ${\mathbf{f}}_i$ linearly to all of $\mathbb{V}$, meaning, for any $\mathbf{v}=\sum _{j=1}^n{\alpha }_j{\mathbf{v}}_j$,

$${\mathbf{f}}_i(\mathbf{v})={\mathbf{f}}_i(\sum _{j=1}^n{\alpha }_j{\mathbf{v}}_j)=\sum _{j=1}^n{\alpha }_j{\mathbf{f}}_i({\mathbf{v}}_j)={\alpha }_i.$$

Then, $\mathcal{F}=\{{\mathbf{f}}_1,\dots ,{\mathbf{f}}_n\}$ form a basis for ${\mathbb{V}}^{\ast }$. The basis $\mathcal{F}$ is called the dual basis of $\mathcal{B}$.

Proof. We first show that ${\mathbf{f}}_1,\dots ,{\mathbf{f}}_n$ are linearly independent.

Let $c_1,\dots ,c_n\in \mathbb{F}$ such that:

$$c_1{\mathbf{f}}_1+\cdots +c_n{\mathbf{f}}_n=\mathbf{0}.$$

Note that the $\mathbf{0}$ in the R.H.S. denotes the zero functional. This means that $c_1{\mathbf{f}}_1+\cdots +c_n{\mathbf{f}}_n$ maps every vector to 0; i.e.,

$$(c_1{\mathbf{f}}_1+\cdots +c_n{\mathbf{f}}_n)(\mathbf{v})=0\mathrm{\forall }\mathbf{v}\in \mathbb{V}.$$

This is valid in particular for the basis vectors in $\mathcal{B}$. Thus,

$$(c_1{\mathbf{f}}_1+\cdots +c_n{\mathbf{f}}_n)({\mathbf{v}}_j)=0\mathrm{\forall }1\le j\le n.$$

But,

$$(c_1{\mathbf{f}}_1+\cdots +c_n{\mathbf{f}}_n)({\mathbf{v}}_j)=\sum _{i=1}^n{c}_i{\mathbf{f}}_i({\mathbf{v}}_j)=c_j$$

from the definition of ${\mathbf{f}}_i$. Thus, $c_j=0$ for $j=1,\dots ,n$. Hence, there is only a trivial linear combination of ${\mathbf{f}}_i$ which equals the zero functional. Thus, $\mathcal{F}$ is a set of linearly independent vectors.

Next, we show that $\mathcal{F}$ spans ${\mathbb{V}}^{\ast }$. Let $\mathbf{f}\in {\mathbb{V}}^{\ast }$. Let

$$b_i=\mathbf{f}({\mathbf{v}}_i)\mathrm{\forall }i=1,\dots ,n$$

be the value of the linear functional $\mathbf{f}$ on the basis vectors in $\mathcal{B}$. We claim that:

$$\mathbf{f}=b_1{\mathbf{f}}_1+\cdots +b_n{\mathbf{f}}_n=\sum _{i=1}^n{b}_i{\mathbf{f}}_i.$$

As shown in Lemma 4.43, if both functionals agree on the basis vectors in $\mathcal{B}$, then they are equal. Now,

$$(b_1{\mathbf{f}}_1+\cdots +b_n{\mathbf{f}}_n)({\mathbf{v}}_i)=\sum _{j=1}^n{b}_j{\mathbf{f}}_j({\mathbf{v}}_i)=b_i=\mathbf{f}({\mathbf{v}}_i)$$

from the definitions of ${\mathbf{f}}_i$ and $b_i$. Thus, the functionals $\mathbf{f}$ and $b_1{\mathbf{f}}_1+\cdots +b_n{\mathbf{f}}_n$ agree on the basis. Thus, they are equal as elements of ${\mathbb{V}}^{\ast }$. Since $\mathbf{f}\in {\mathbb{V}}^{\ast }$ was arbitrary, hence, $\mathcal{F}$ spans ${\mathbb{V}}^{\ast }$.

Since functionals in $\mathcal{F}$ are linearly independent and they span ${\mathbb{V}}^{\ast }$, they form a basis of ${\mathbb{V}}^{\ast }$.

Corollary 4.15 (Dimension of the dual space)

If $\mathbb{V}$ is finite dimensional, then its dual space ${\mathbb{V}}^{\ast }$ is also finite dimensional and

$$\dim \mathbb{V}=\dim {\mathbb{V}}^{\ast }.$$

Theorem 4.96

Let $\mathbb{V}$ be finite dimensional. Let $\mathbf{v}\in \mathbb{V}$ such that $\mathbf{f}(\mathbf{v})=0$ for every $\mathbf{f}\in {\mathbb{V}}^{\ast }$. Then $\mathbf{v}=\mathbf{0}$.

Proof. Let $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be a basis of $\mathbb{V}$. Let $\mathcal{F}=\{{\mathbf{f}}_1,\dots ,{\mathbf{f}}_n\}$ be the dual basis of ${\mathbb{V}}^{\ast }$ with ${\mathbf{f}}_i({\mathbf{v}}_j)=\delta (i,j)$.

$\mathbf{v}$ can be written as as the linear combination of the basis vectors:

$$\mathbf{v}=a_1{\mathbf{v}}_1+\cdots +a_n{\mathbf{v}}_n.$$

By hypothesis ${\mathbf{f}}_i(\mathbf{v})=0$. Thus,

$${\mathbf{f}}_i(\mathbf{v})=\sum _{j=1}^n{a}_i{\mathbf{f}}_i({\mathbf{v}}_j)=a_i=0.$$

Thus, $a_i=0$ for all $i=1,\dots ,n$. Thus,

$$\mathbf{v}=\sum _{i=1}^n{a}_i{\mathbf{v}}_i=\mathbf{0}.$$

4.6.4. Null Space of Linear Functionals

We shall describe the null space of a linear functional and show that for finite dimensional vector spaces, its codimension is 1.

Theorem 4.97 (Null space/kernel of a linear functional)

If $\mathbf{f}$ is a linear functional on $\mathbb{V}$, then

$${\mathbf{f}}^{-1}(0)=\{\mathbf{x}\in \mathbb{V}|\mathbf{f}(\mathbf{v})=0\}$$

is a linear subspace of $\mathbb{V}$.

Proof. We know that $0$ is the zero vector of the vector space $\mathbb{F}$. Thus, ${\mathbf{f}}^{-1}(0)$ is nothing but the null space of $\mathbf{f}$ and hence it is a linear subspace of $\mathbb{V}$.

Another proof from first principles:

1. Let $\mathbf{u},\mathbf{v}\in {\mathbf{f}}^{-1}(0)$.

2. Then, $\mathbf{f}(\alpha \mathbf{u}+\mathbf{v})=\alpha \mathbf{f}(\mathbf{u})+\mathbf{f}(\mathbf{v})=0$.

3. Thus, $\alpha \mathbf{u}+\mathbf{v}\in {\mathbf{f}}^{-1}(0)$.

4. Hence, ${\mathbf{f}}^{-1}(0)$ is a subspace.

Theorem 4.98

Let $\mathbf{f}$ be a nonzero linear functional on $\mathbb{V}$. Let $\mathbf{a}\in \mathbb{V}$ be such that $\mathbf{f}(\mathbf{a})\ne 0$. Then, any point $\mathbf{p}\in \mathbb{V}$ can be written uniquely as:

$$\mathbf{p}=\lambda \mathbf{a}+\mathbf{x}$$

where $\lambda \in \mathbb{F}$ and $\mathbf{x}\in {\mathbf{f}}^{-1}(0)$.

In this representation, $\mathbf{f}$ and $\mathbf{a}$ are fixed while $\lambda$ and $\mathbf{x}$ are allowed to vary.

Proof. Let $\mathbf{p}\in \mathbb{V}$ be arbitrary. We wish to construct a representation:

$$\mathbf{p}=\lambda \mathbf{a}+\mathbf{x}$$

such that $\mathbf{x}\in {\mathbf{f}}^{-1}(0)$ and $\lambda \in \mathbb{F}$.

Applying $\mathbf{f}$ on both sides, we get:

$$\mathbf{f}(\mathbf{p})=\lambda \mathbf{f}(\mathbf{a})+\mathbf{f}(\mathbf{x})=\lambda \mathbf{f}(\mathbf{a})$$

since $\mathbf{x}\in {\mathbf{f}}^{-1}(0)$.

This gives us:

$$\lambda =\frac{\mathbf{f}(\mathbf{p})}{\mathbf{f}(\mathbf{a})}$$

since we know that $\mathbf{f}(\mathbf{a})\ne 0$.

Putting this particular value of $\lambda$, we get:

$$\mathbf{x}=\mathbf{p}-\frac{\mathbf{f}(\mathbf{p})}{\mathbf{f}(\mathbf{a})}\mathbf{a}.$$

Thus, for every $\mathbf{p}\in \mathbb{V}$, we can find $\lambda =\frac{\mathbf{f}(\mathbf{p})}{\mathbf{f}(\mathbf{a})}$ and $\mathbf{x}=\mathbf{p}-\frac{\mathbf{f}(\mathbf{p})}{\mathbf{f}(\mathbf{a})}\mathbf{a}$ such that $\mathbf{f}(\mathbf{x})=0$ and:

$$\mathbf{p}=\lambda \mathbf{a}+\mathbf{x}.$$

We have established existence of this representation. Next, we establish that this representation is unique.

Suppose their was another representation:

$$\mathbf{p}=\mu \mathbf{a}+\mathbf{y}$$

with $\mathbf{f}(\mathbf{y})=0$ and $\mu \in \mathbb{F}$.

Then, we would have:

$$\mathbf{f}(\mu \mathbf{a}+\mathbf{y})=\mu \mathbf{f}(\mathbf{a})=\mathbf{f}(\lambda \mathbf{a}+\mathbf{x})=\lambda \mathbf{f}(\mathbf{a}).$$

Since $\mathbf{f}(\mathbf{a})\ne 0$, hence $\lambda =\mu$ must hold. Therefore, $\mathbf{x}=\mathbf{y}$ must hold true too. Hence, the representation is unique.

Theorem 4.99 (Dimension of the kernel of a linear functional)

Let $\mathbb{V}$ be a finite dimensional vector space of dimension $n$. Then, the dimension of the null space of any nonzero linear functional on $\mathbb{V}$ is $n-1$. In other words, the codimension of the kernel of a nonzero linear functional is 1.

Proof. Let $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_k\}$ the any basis for the space ${\mathbf{f}}^{-1}(0)$.

Choose any $\mathbf{a}\in \mathbb{V}$ such that $\mathbf{f}(\mathbf{a})\ne 0$.

Then, the set:

$$\mathcal{C}=\{\mathbf{a},{\mathbf{v}}_1,\dots ,{\mathbf{v}}_k\}$$

is linearly independent since $\mathbf{a}\notin {\mathbf{f}}^{-1}(0)$.

At the same time, since any vector $\mathbf{x}\in \mathbb{V}$ can be represented using $\mathbf{a}$ and the null space spanned by $\mathcal{B}$, hence $\mathcal{C}$ spans $\mathbb{V}$ (while $\mathcal{B}$ doesn’t). Thus,

$$n=k+1.$$

Alternatively:

$$\dim {\mathbf{f}}^{-1}(0)=k=n-1$$

or

$$\mathrm{codim}{\mathbf{f}}^{-1}(0)=n-\dim {\mathbf{f}}^{-1}(0)=1.$$

4.6.5. Hyper Planes

We present a general definition of hyperplanes in this section. For a definition specific to real vector spaces, see Definition 9.1.

Definition 4.87 (Hyperplane)

Let $\mathbf{f}$ be a nonzero linear functional on $\mathbb{V}$. Let $a\in \mathbb{F}$. A set of the form:

$$H_{\mathbf{f},a}\triangleq \{\mathbf{x}\in \mathbb{V}|\mathbf{f}(\mathbf{x})=a\}$$

is called a hyperplane. In other words,

$$H_{\mathbf{f},a}={\mathbf{f}}^{-1}(a).$$

Theorem 4.100 (A span as a hyperplane)

Let $\mathbb{V}$ be a finite dimensional space with $\dim \mathbb{V}=n$. Let $S=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_{n-1}\}$ be any set of $n-1$ linearly independent vectors of $\mathbb{V}$. Then, $\mathrm{span}S$ is a hyperplane.

Proof. Since $\dim \mathbb{V}=n$, it is possible to select a vector ${\mathbf{v}}_n$ such that $\mathcal{B}=S\cup \{{\mathbf{v}}_n\}$ is linearly independent and forms a basis for $\mathbb{V}$.

Following Theorem 4.95, it is possible to construct a linear functional $\mathbf{f}$ such that:

$$\begin{array}{r}\mathbf{f}({\mathbf{v}}_i)=\{\begin{array}{lllll}1& & \text{ if }& & i=n\\ 0& & \text{ if }& & 1\le i<n\end{array}.\end{array}$$

Now, consider the null space of $\mathbf{f}$ given by ${\mathbf{f}}^{-1}(0)$. It is clear by definition that ${\mathbf{v}}_i\in {\mathbf{f}}^{-1}(0)$ for $i=1,\dots ,n-1$.

From Theorem 4.99, $\dim {\mathbf{f}}^{-1}(0)=n-1$. Thus, $S$ forms a basis for ${\mathbf{f}}^{-1}(0)$ and:

$$\mathrm{span}S={\mathbf{f}}^{-1}(0).$$

Thus, $\mathrm{span}S$ is a hyperplane.

4.6.6. Dual Space of Dual Space

Since ${\mathbb{V}}^{\ast }$ (the dual of $\mathbb{V}$) is a vector space, we can think of the dual space of ${\mathbb{V}}^{\ast }$ too. The dual of the dual is denoted by ${\mathbb{V}}^{\ast \ast }$. Consider the case where $\mathbb{V}$ is finite dimensional.

1. $\mathbb{V}$ and ${\mathbb{V}}^{\ast }$ have the same dimension.

2. Thus, $\mathbb{V}$ and ${\mathbb{V}}^{\ast }$ are isomorphic.

3. Since ${\mathbb{V}}^{\ast }$ is finite dimensional, hence ${\mathbb{V}}^{\ast \ast }$ is finite dimensional too.

4. In fact, $\dim {\mathbb{V}}^{\ast }=\dim {\mathbb{V}}^{\ast \ast }$.

5. Thus, ${\mathbb{V}}^{\ast }$ and ${\mathbb{V}}^{\ast \ast }$ are isomorphic.

6. Hence, $\mathbb{V}$ and ${\mathbb{V}}^{\ast \ast }$ are isomorphic too.

It is possible to write an isomorphism between $\mathbb{V}$ and ${\mathbb{V}}^{\ast \ast }$ without the choice of a basis. Such an isomorphism is said to be natural.

4.6.7. Normed Vector Spaces

If $\mathbb{V}$ is endowed with a norm $\Vert \cdot \Vert$, it makes sense to come up with the notion of a norm on the dual space too. The dual norm is a measure of size of the linear functional. By size of a functional, we mean how big is the number $|f(\mathbf{x})|$ with respect to the size of $\mathbf{x}$ given by $\Vert \mathbf{x}\Vert$.

Definition 4.88 (Dual norm)

Let $\mathbb{V}$ be a normed linear space endowed with a norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$ and ${\mathbb{V}}^{\ast }$ be its dual space of linear functionals. The dual norm of a linear functional $\mathbf{f}$ belonging to ${\mathbb{V}}^{\ast }$, denoted by $\Vert \cdot {\Vert }_{\ast }:{\mathbb{V}}^{\ast }\to \mathbb{R}$, is defined as:

$$\Vert \mathbf{f}{\Vert }_{\ast }\triangleq \underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\mathbf{f}(\mathbf{x})|\}.$$

Theorem 4.101 (Dual norm is a norm)

The dual norm $\Vert \cdot {\Vert }_{\ast }:{\mathbb{V}}^{\ast }\to \mathbb{R}$ as defined in Definition 4.88 is indeed a norm.

Proof. [Positive definiteness]

For the zero functional, $\mathbf{0}(\mathbf{x})=0\mathrm{\forall }\mathbf{x}\in \mathbb{V}$. Thus, $\Vert \mathbf{0}{\Vert }_{\ast }=0$.

For the converse, we proceed as follows:

1. For any other functional $\mathbf{f}$, there exists a nonzero $\mathbf{u}\in \mathbb{V}$ such that $\mathbf{f}(\mathbf{u})\ne 0$.

2. Let $\mathbf{x}=\frac{\mathbf{u}}{\Vert \mathbf{u}\Vert }$.

3. Then $\Vert \mathbf{x}\Vert =1$.

4. Also, $\mathbf{f}(\mathbf{x})=\frac{\mathbf{f}(\mathbf{u})}{\Vert \mathbf{u}\Vert }\ne 0$.

5. Thus,

   $$\Vert \mathbf{f}{\Vert }_{\ast }=\underset{\Vert \mathbf{y}\Vert \le 1}{sup}\{|\mathbf{f}(\mathbf{y})|\}\ge |\mathbf{f}(\mathbf{x})|>0.$$

6. Thus, $\Vert \mathbf{f}{\Vert }_{\ast }>0$ if $\mathbf{f}\ne \mathbf{0}$.

[Positive homogeneity]

$$\begin{array}{rl}\Vert \alpha \mathbf{f}{\Vert }_{\ast }& =\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|(\alpha \mathbf{f})(\mathbf{x})|\}\\ & =\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\alpha \mathbf{f}(\mathbf{x})|\}\\ & =\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\alpha ||\mathbf{f}(\mathbf{x})|\}\\ & =|\alpha |\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\mathbf{f}(\mathbf{x})|\}\\ & =|\alpha |\Vert \mathbf{f}{\Vert }_{\ast }.\end{array}$$

[Triangle inequality]

Let $\mathbf{f},\mathbf{g}\in {\mathbb{V}}^{\ast }$ be two linear functionals.

$$\begin{array}{rl}\Vert \mathbf{f}+\mathbf{g}{\Vert }_{\ast }& =\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|(\mathbf{f}+\mathbf{g})(\mathbf{x})|\}\\ & =\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\mathbf{f}(\mathbf{x})+\mathbf{g}(\mathbf{x})|\}\\ & \le \underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\mathbf{f}(\mathbf{x})|+|\mathbf{g}(\mathbf{x})|\}\\ & \le \underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\mathbf{f}(\mathbf{x})|\}+\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|\mathbf{g}(\mathbf{x})|\}\\ & =\Vert \mathbf{f}{\Vert }_{\ast }+\Vert \mathbf{g}{\Vert }_{\ast }.\end{array}$$

4.6.8. Inner Product Spaces

The inner product is a binary operator from $\mathbb{V}\times \mathbb{V}$ to $\mathbb{F}$. If the second argument is fixed, then it becomes a linear functional.

Theorem 4.102 (Inner product as linear functional)

Let $\mathbb{V}$ be an inner product space. For any $\mathbf{v}\in \mathbb{V}$, the mapping $T_{\mathbf{v}}:\mathbb{V}\to \mathbb{F}$ defined by:

$$T_{\mathbf{v}}(\mathbf{x})\triangleq ⟨\mathbf{x},\mathbf{v}⟩\mathrm{\forall }\mathbf{x}\in \mathbb{V}$$

is a linear functional. Consequently, $T_{\mathbf{v}}\in {\mathbb{V}}^{\ast }$.

Proof. Since an inner product is linear in the first argument, hence:

$$T_{\mathbf{v}}(\alpha \mathbf{x}+\mathbf{y})=⟨\alpha \mathbf{x}+\mathbf{y},\mathbf{v}⟩=\alpha ⟨\mathbf{x},\mathbf{v}⟩+⟨\mathbf{y},\mathbf{v}⟩=\alpha T_{\mathbf{v}}(\mathbf{x})+T_{\mathbf{v}}(\mathbf{y}).$$

Thus, the functional $T_{\mathbf{v}}$ is linear.

Theorem 4.103 (Arithmetic on linear functionals)

Addition of inner product based linear functionals:

$$T_{\mathbf{x}}+T_{\mathbf{y}}=T_{\mathbf{x}+\mathbf{y}}.$$

Scalar multiplication on inner product based linear functions:

$$\alpha T_{\mathbf{x}}=T_{\bar{\alpha }\mathbf{x}}.$$

Proof. Recall that sum of functions is defined as:

$$(T_{\mathbf{x}}+T_{\mathbf{y}})(\mathbf{v})=T_{\mathbf{x}}(\mathbf{v})+T_{\mathbf{y}}(\mathbf{v}).$$

Now,

$$T_{\mathbf{x}+\mathbf{y}}(\mathbf{v})=⟨\mathbf{v},\mathbf{x}+\mathbf{y}⟩=⟨\mathbf{v},\mathbf{x}⟩+⟨\mathbf{v},\mathbf{y}⟩=T_{\mathbf{x}}(\mathbf{v})+T_{\mathbf{y}}(\mathbf{v})=(T_{\mathbf{x}}+T_{\mathbf{y}})(\mathbf{v})$$

Thus, $T_{\mathbf{x}+\mathbf{y}}=T_{\mathbf{x}+\mathbf{y}}$.

Recall that scaling of a function is defined as:

$$(\alpha T_{\mathbf{x}})(\mathbf{v})=\alpha (T_{\mathbf{x}}(\mathbf{v})).$$

Now,

$$\alpha (T_{\mathbf{x}}(\mathbf{v}))=\alpha ⟨\mathbf{v},\mathbf{x}⟩=⟨\mathbf{v},\bar{\alpha }\mathbf{x}⟩=T_{\bar{\alpha }\mathbf{x}}(\mathbf{v}).$$

Thus, $\alpha T_{\mathbf{x}}=T_{\bar{\alpha }\mathbf{x}}$.

Theorem 4.104 (Linear functional as inner product)

Let $\mathbb{V}$ be a finite dimensional inner product space and let $\mathbf{f}$ be a linear functional in ${\mathbb{V}}^{\ast }$. Then, there exists a vector $\mathbf{v}\in \mathbb{V}$ such that $\mathbf{f}=T_{\mathbf{v}}$.

In other words, every linear functional is an inner product.

Proof. Recall from Theorem 4.82 that every finite dimensional inner product space has an orthonormal basis.

Let $\mathcal{B}=\{{\mathbf{e}}_1,\dots ,{\mathbf{e}}_n\}$ be an orthonormal basis of $\mathbb{V}$.

Now, consider the corresponding linear functionals $T_{{\mathbf{e}}_j}$ for $j=1,\dots ,n$.

Note that:

$$T_{{\mathbf{e}}_j}({\mathbf{e}}_i)=⟨{\mathbf{e}}_i,{\mathbf{e}}_j⟩=\delta (i,j)$$

since $\mathcal{B}$ is an orthonormal basis.

Then, following Theorem 4.95, the set $\mathcal{F}=\{T_{{\mathbf{e}}_1},\dots ,T_{{\mathbf{e}}_n}\}$ forms a basis ${\mathbb{V}}^{\ast }$.

Thus, any $\mathbf{f}\in {\mathbb{V}}^{\ast }$ can be written as:

$$\mathbf{f}=\sum _{j=1}^n{c}_j{T}_{{\mathbf{e}}_j}=\sum _{j=1}^n{T}_{\overline{c_j}{\mathbf{e}}_j}=T_{\sum _{j=1}^n\overline{c_j}{\mathbf{e}}_j}$$

using Theorem 4.103 above.

Thus, $\mathbf{f}$ is equal to an inner product by the vector $\sum _{j=1}^n\overline{c_j}{\mathbf{e}}_j$.

Theorem 4.105 (Zero functional in inner product space)

Let $\mathbb{V}$ be an inner product space. Then, $\mathbf{0}\in {\mathbb{V}}^{\ast }$ is same as $T_{\mathbf{0}}=⟨\cdot ,\mathbf{0}⟩$.

In other words,

$$T_{\mathbf{0}}=\mathbf{0}\in {\mathbb{V}}^{\ast }$$

and there is no other $\mathbf{v}\in V$ such that $T_{\mathbf{v}}=\mathbf{0}$.

Proof. For $\mathbf{0}\in \mathbb{V}$, it is straight-forward to see that

$$T_{\mathbf{0}}(\mathbf{v})=⟨\mathbf{v},\mathbf{0}⟩=0.$$

Thus, $T_{\mathbf{0}}$ is the zero functional. To show that there is no other $\mathbf{v}\in \mathbb{V}$ such that $T_{\mathbf{v}}$ is a zero functional, we note that,

$$\mathbf{v}\ne 0⟹⟨\mathbf{v},\mathbf{v}⟩>0.$$

Thus, $T_{\mathbf{v}}(\mathbf{v})>0$ if $\mathbf{v}\ne \mathbf{0}$.

Theorem 4.106 (Isomorphism between $\mathbb{V}$ and ${\mathbb{V}}^{\ast }$)

Let $\mathbb{V}$ be an inner product space. Define a map $T:\mathbb{V}\to {\mathbb{V}}^{\ast }$ by

$$T(\mathbf{v})\triangleq ⟨\cdot ,\mathbf{v}⟩=T_{\mathbf{v}};$$

i.e., $T_{\mathbf{v}}$ is a linear functional on $\mathbb{V}$ whose value on $\mathbf{x}\in \mathbb{V}$ is $⟨\mathbf{x},\mathbf{v}⟩$.

Then, $T$ is an isomorphism.

Proof. To show that $T$ is an isomorphism, we need to show that:

• $T$ is injective.

• $T$ is surjective.

• $T$ preserves the vector space algebraic structure.

Theorem 4.107 (Dual norm in an inner product space)

Let $\mathbb{V}$ be an inner product space endowed with a norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$.

Let $\mathbf{v}\in \mathbb{V}$ correspond to a linear functional $T_{\mathbf{v}}\in {\mathbb{V}}^{\ast }$. Then, its dual norm is given by:

$$\Vert \mathbf{v}{\Vert }_{\ast }=\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{|⟨\mathbf{x},\mathbf{v}⟩|\}.$$

Note that in this identity, $\Vert \cdot \Vert$ is any arbitrary norm on $\mathbb{V}$. It need not be the norm induced by the inner product.

Proof. This version of dual norm comes from the fact that:

$$T_{\mathbf{v}}(\mathbf{x})=⟨\mathbf{x},\mathbf{v}⟩\mathrm{\forall }\mathbf{x}\in \mathbb{V}.$$

Plugging it into the equation in Definition 4.88, we get the desired identity.

Theorem 4.108 (Generalized Cauchy Schwartz inequality)

Let $\mathbb{V}$ be an inner product space endowed with a norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$.

For any $\mathbf{u}\in \mathbb{V}$, $\mathbf{v}\in {\mathbb{V}}^{\ast }$:

$$|⟨\mathbf{u},\mathbf{v}⟩|\le \Vert \mathbf{u}\Vert \Vert \mathbf{v}{\Vert }_{\ast }.$$

Proof. If $\mathbf{u}=\mathbf{0}$, the inequality holds. Assume $\mathbf{u}\ne \mathbf{0}$.

1. Let $\mathbf{x}=\frac{\mathbf{u}}{\Vert \mathbf{u}\Vert }$.

2. Then, $\Vert \mathbf{x}\Vert =1$.

3. By Theorem 4.107

   $$\Vert \mathbf{v}{\Vert }_{\ast }\ge |⟨\mathbf{x},\mathbf{v}⟩|=\left|⟨\frac{\mathbf{u}}{\Vert \mathbf{u}\Vert },\mathbf{v}⟩\right|=\frac{|⟨\mathbf{u},\mathbf{v}⟩|}{\Vert \mathbf{u}\Vert }.$$

4. Multiplying both sides by $\Vert \mathbf{u}\Vert$, we get:

   $$\Vert \mathbf{u}\Vert \Vert \mathbf{v}{\Vert }_{\ast }\ge |⟨\mathbf{u},\mathbf{v}⟩|.$$

4.6.9. Real Inner Product Spaces

In this section, we assume that $\mathbb{V}$ is a vector space over $\mathbb{R}$. Some of the expressions simplify further for real vector spaces.

Theorem 4.109 (Dual norm in a real inner product space)

Let $\mathbb{V}$ be a real inner product space endowed with a norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$.

Let $\mathbf{v}\in \mathbb{V}$ correspond to a linear functional $T_{\mathbf{v}}$. Then, its dual norm is given by:

$$\Vert \mathbf{v}{\Vert }_{\ast }=\underset{\Vert \mathbf{x}\Vert \le 1}{sup}\{⟨\mathbf{x},\mathbf{v}⟩\}.$$

Proof. Assuming $\mathbb{V}$ to be a real vector space, we note that

1. $\Vert \mathbf{x}\Vert =\Vert -\mathbf{x}\Vert$.

2. Thus, $\Vert \mathbf{x}\Vert \le 1⟺\Vert -\mathbf{x}\Vert \le 1$.

3. Also $⟨\mathbf{x},\mathbf{v}⟩=-⟨-\mathbf{x},\mathbf{v}⟩$.

4. Either $⟨\mathbf{x},\mathbf{v}⟩$ or $⟨-\mathbf{x},\mathbf{v}⟩$ is nonnegative.

5. Thus, the identity in Theorem 4.107 simplifies to this version.