4.4. Normed Linear Spaces

A norm is a real number attached to every vector. Norm is a generalization of the notion of length. Adding a norm to a vector space makes it a normed linear space with rich topological properties as a norm induces a distance function (a metric) on the vector space converting it into a metric space with an algebraic structure. We can identify open sets, closed sets, bounded sets, compact sets, etc.. We can introduce the notion of continuity on the functions defined from one normed linear space to another metric space (which could be a normed linear space or just the real line).

The algebraic structure allows us to consider operations like translations, scaling, rotation, and general affine transformations on sets of vectors which can be thought of as geometrical objects.

There are many ways to define norms on a vector space. A question arises as to which norms are equivalent to each other in the sense that they determine the same topology on the underlying vector space. It turns out that in finite dimensional vector spaces, all norms are equivalent.

Linear transformations play a central role in linear algebra. Linear transformations are generally unbounded by the usual notion of boundedness of a function (where we say that the range of the function is bounded). However, there is another way to examine the boundedness of linear transformations by examine the ratio of norms of the input to the transformation and the output generated by it. For bounded linear transformations, the ratio is bounded. For matrices, this is the largest singular value. Not all linear transformations are continuous, but linear transformations on finite dimensional spaces are continuous. In fact, for linear transformations, boundedness, continuity, uniform continuity and Lipschitz continuity turn out to be the same thing.

A normed linear space that is complete is called a Banach space.

We restrict our attention to real vector spaces and complex vector spaces. Thus, the field $\mathbb{F}$ can be either $\mathbb{R}$ or $\mathbb{C}$.

4.4.1. Norm

Definition 4.59 (Norm)

A norm over a $\mathbb{F}$-vector space $\mathbb{V}$ is any real valued function $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$ mapping $\mathbf{v}\mapsto \Vert \mathbf{v}\Vert$ satisfying following properties:

1. [Positive definiteness]

   $$\Vert \mathbf{v}\Vert \ge 0\mathrm{\forall }\mathbf{v}\in \mathbb{V}\text{ and }\Vert \mathbf{v}\Vert =0⟺\mathbf{v}=\mathbf{0}.$$

2. [Positive homogeneity]

   $$\Vert \alpha \mathbf{v}\Vert =|\alpha |\Vert \mathbf{v}\Vert \mathrm{\forall }\alpha \in \mathbb{F};\mathrm{\forall }\mathbf{v}\in \mathbb{V}.$$

3. [Triangle inequality]

   $$\Vert {\mathbf{v}}_1+{\mathbf{v}}_2\Vert \le \Vert {\mathbf{v}}_1\Vert +\Vert {\mathbf{v}}_2\Vert \mathrm{\forall }{\mathbf{v}}_1,{\mathbf{v}}_2\in \mathbb{V}.$$

The norm doesn’t change on negation.

Remark 4.5 (Negation and norm)

$$\Vert -\mathbf{v}\Vert =\Vert (-1)\mathbf{v}\Vert =|-1|\Vert \mathbf{v}\Vert =\Vert \mathbf{v}\Vert \mathrm{\forall }\mathbf{v}\in \mathbb{V}.$$

There is a different form of triangle inequality which is quite useful in the following analysis. It can be derived from the properties of a norm.

Theorem 4.33 (Triangle inequality II)

$$|\Vert \mathbf{x}\Vert -\Vert \mathbf{y}\Vert |\le \Vert \mathbf{x}-\mathbf{y}\Vert \mathrm{\forall }\mathbf{x},\mathbf{y}\in \mathbb{V}.$$

Proof. Putting ${\mathbf{v}}_1=\mathbf{x}$ and ${\mathbf{v}}_2=\mathbf{y}-\mathbf{x}$ in the triangle inequality, we get:

$$\begin{array}{r}\begin{array}{rl}& \Vert \mathbf{x}+\mathbf{y}-\mathbf{x}\Vert \le \Vert \mathbf{x}\Vert +\Vert \mathbf{y}-\mathbf{x}\Vert \\ & ⟺\Vert \mathbf{y}\Vert \le \Vert \mathbf{x}\Vert +\Vert \mathbf{y}-\mathbf{x}\Vert \\ & ⟺\Vert \mathbf{y}\Vert -\Vert \mathbf{x}\Vert \le \Vert \mathbf{y}-\mathbf{x}\Vert =\Vert \mathbf{x}-\mathbf{y}\Vert .\end{array}\end{array}$$

Interchanging $\mathbf{x}$ and $\mathbf{y}$ in previous inequality, we get:

$$\Vert \mathbf{x}\Vert -\Vert \mathbf{y}\Vert \le \Vert \mathbf{x}-\mathbf{y}\Vert .$$

Combining the two inequalities, we get:

$$|\Vert \mathbf{x}\Vert -\Vert \mathbf{y}\Vert |\le \Vert \mathbf{x}-\mathbf{y}\Vert \mathrm{\forall }\mathbf{x},\mathbf{y}\in \mathbb{V}.$$

Theorem 4.34 (Triangle inequality for vector differences)

The triangle inequality is equivalent to the following property:

$$\Vert \mathbf{x}-\mathbf{y}\Vert \le \Vert \mathbf{x}-\mathbf{z}\Vert +\Vert \mathbf{z}-\mathbf{y}\Vert \mathrm{\forall }\mathbf{x},\mathbf{y},\mathbf{z}\in \mathbb{V}.$$

Proof. Start with

$$\Vert {\mathbf{v}}_1+{\mathbf{v}}_2\Vert \le \Vert {\mathbf{v}}_1\Vert +\Vert {\mathbf{v}}_2\Vert .$$

Put ${\mathbf{v}}_1=\mathbf{x}-\mathbf{z}$ and ${\mathbf{v}}_2=\mathbf{z}-\mathbf{y}$. We get:

$$\Vert \mathbf{x}-\mathbf{z}+\mathbf{z}-\mathbf{y}\Vert \le \Vert \mathbf{x}-\mathbf{z}\Vert +\Vert \mathbf{z}-\mathbf{y}\Vert ⟺\Vert \mathbf{x}-\mathbf{y}\Vert \le \Vert \mathbf{x}-\mathbf{z}\Vert +\Vert \mathbf{z}-\mathbf{y}\Vert .$$

For the converse, start with:

$$\Vert \mathbf{x}-\mathbf{y}\Vert \le \Vert \mathbf{x}-\mathbf{z}\Vert +\Vert \mathbf{z}-\mathbf{y}\Vert$$

Put ${\mathbf{v}}_1=\mathbf{x}-\mathbf{z}$ and ${\mathbf{v}}_2=\mathbf{z}-\mathbf{y}$. Then, ${\mathbf{v}}_1+{\mathbf{v}}_2=\mathbf{x}-\mathbf{y}$. We get:

$$\Vert {\mathbf{v}}_1+{\mathbf{v}}_2\Vert \le \Vert {\mathbf{v}}_1\Vert +\Vert {\mathbf{v}}_2\Vert .$$

4.4.2. Normed Linear Space

Definition 4.60 (Normed linear space)

An $\mathbb{F}$-vector space $\mathbb{V}$ equipped with a norm $\Vert \Vert :\mathbb{V}\to \mathbb{R}$ is known as a normed linear space. Other common terms are normed vector space or simply normed space.

Example 4.15 (Norm for the trivial vector space)

The trivial vector space contains a single vector which is the zero vector; i.e., $\mathbb{V}=\{\mathbf{0}\}$.

A function $f:\mathbb{V}\to \mathbb{R}$ given by $f(\mathbf{0})=0$ satisfies all the properties of a norm. It is easy to see that this is the only possible norm for the trivial vector space.

Remark 4.6

We will assume that the vector space is non-trivial; i.e., different from $\{\mathbf{0}\}$ in this section. Wherever necessary, we will provide details about how a particular result is valid for the trivial vector space also.

Example 4.16 (Real line $\mathbb{R}$)

Recall from Example 4.3 that $\mathbb{R}$ is a vector space by itself.

The function $f:\mathbb{R}\to \mathbb{R}$ given by

$$f(x)=|x|$$

satisfies all the requirements of a norm. Thus, $(\mathbb{R},|\cdot |)$ is a normed linear space.

Example 4.17 (Euclidean space ${\mathbb{R}}^n$)

The Euclidean space ${\mathbb{R}}^n$ can be equipped with a variety of norms. They are discussed in detail in The Euclidean Space.

1. The Euclidean norm

2. ${\ell }_p$ norms

When we propose a norm for a vector space, we justify it by showing that it satisfies all the properties of a norm.

Theorem 4.111 justifies that all ${\ell }_p$ norms on ${\mathbb{R}}^n$ are indeed norms.

4.4.3. Normed Space as a Metric Space

Definition 4.61 (Metric induced by a norm)

Every norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$ induces a metric defined as:

$$d(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}\Vert \mathrm{\forall }\mathbf{x},\mathbf{y}\in \mathbb{V}.$$

Theorem 4.35 (Norm induced metric justification)

The metric $d:\mathbb{V}\times \mathbb{V}\to \mathbb{R}$ induced by a norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$ is indeed a metric satisfying all the properties of a metric( distance function).

Proof. We proceed as follows:

1. Non-negativity: $d(\mathbf{x},\mathbf{y})\ge 0$ since $\Vert \cdot \Vert$ is positive definite.

2. Identity of indiscernibles.

   1. Assume $d(\mathbf{x},\mathbf{y})=0$.

   2. Then, $\Vert \mathbf{x}-\mathbf{y}\Vert =0$.

   3. Thus, $\mathbf{x}=\mathbf{y}$ since $\Vert \cdot \Vert$ is positive definite.

   4. Now, assume $\mathbf{x}=\mathbf{y}$.

   5. Then, $d(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}\Vert =\Vert \mathbf{0}\Vert =0$ since $\Vert \cdot \Vert$ is positive definite.

3. Symmetry: $d(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}\Vert =\Vert (-1)(\mathbf{y}-\mathbf{x})\Vert =|-1|\Vert \mathbf{y}-\mathbf{x}\Vert =d(\mathbf{y},\mathbf{x})$ using the positive homogeneity property of $\Vert \cdot \Vert$.

4. Triangle inequality: See Theorem 4.34 above.

Remark 4.7

We will use the notation $\Vert \cdot \Vert$ to denote both the norm and the metric induced by the norm.

Definition 4.62 (Metric space from a norm)

The normed space $\mathbb{V}$ equipped with the metric induced by the norm as defined in Definition 4.61 becomes a metric space $(\mathbb{V},\Vert \cdot \Vert )$.

If the norm and the induced metric are clear from the context, then we shall simply write it as $\mathbb{V}$.

Theorem 4.36 (Translation invariance)

The metric induced by a norm is translation invariant.

For any $\mathbf{u},\mathbf{v},\mathbf{w}\in \mathbb{V}$:

$$d(\mathbf{u},\mathbf{v})=d(\mathbf{u}+\mathbf{w},\mathbf{v}+\mathbf{w}).$$

Proof. Expanding from definition:

$$d(\mathbf{u}+\mathbf{w},\mathbf{v}+\mathbf{w})=\Vert \mathbf{u}+\mathbf{w}-(\mathbf{v}+\mathbf{w})\Vert =\Vert \mathbf{u}-\mathbf{v}\Vert =d(\mathbf{u},\mathbf{v}).$$

The topology of a general metric space is discussed in detail in Metric Topology and sections thereafter. In this section, we discuss results which are specific to normed linear spaces as they take advantage of the additional structure provided by the vector space.

1. There is a special zero vector $\mathbf{0}\in \mathbb{V}$. It provides a reference point to define unit balls.

2. Vectors in $\mathbb{V}$ can be added, subtracted and scaled. Thus, general balls can be described in terms of unit balls.

3. It is possible to introduce the notion of translation of sets. Recall that the metric induced by the norm is translation invariant.

4. Sets of vectors in a vector space can be added/subtracted/scaled since the underlying vectors can be. This produces a number of interesting phenomena.

4.4.4. Balls

An open ball in a normed space is defined analogously as:

$$B(\mathbf{a},r)=\{\mathbf{x}\in \mathbb{V}|\Vert \mathbf{x}-\mathbf{a}\Vert <r\}.$$

A closed ball in a normed space is defined analogously as:

$$B[\mathbf{a},r]=\{\mathbf{x}\in \mathbb{V}|\Vert \mathbf{x}-\mathbf{a}\Vert \le r\}.$$

We sometimes use the notation $B_{\Vert \cdot \Vert }(\mathbf{a},r)$ and $B_{\Vert \cdot \Vert }[\mathbf{a},r]$ to identify the specific norm being used to describe the open and closed balls.

Definition 4.63 (Unit ball)

A ball centered at origin $\mathbf{0}\in \mathbb{V}$ is called a unit ball if its radius is $1$. $B[\mathbf{0},1]$ denotes a closed unit ball and $B(\mathbf{0},1)$ denotes an open unit ball. The open unit ball is often written simply as $B$.

Observation 4.1 (Ball arithmetic)

Following the notation in Definition 4.25, a closed ball can be expressed in terms of closed unit ball as:

$$B[\mathbf{x},r]=\mathbf{x}+rB[\mathbf{0},1].$$

Similarly, any open ball can be expressed in terms of the open unit ball as:

$$B(\mathbf{x},r)=\mathbf{x}+rB(\mathbf{0},1).$$

Theorem 4.37 (Ball addition and subtraction)

Let $\mathbb{V}$ be a normed linear space and $B$ be the unit open ball. Then,

$$B+B=B-B=2B.$$

Proof. Due to Theorem 4.16, $2B\subseteq B+B$.

We now show that $B+B\subseteq 2B$.

1. Let $\mathbf{a}\in B+B$.

2. Then, $\mathbf{a}=\mathbf{b}+\mathbf{c}$ such that $\mathbf{b}\in B$ and $\mathbf{c}\in B$.

3. Now,

   $\Vert \mathbf{a}\Vert =\Vert \mathbf{b}+\mathbf{c}\Vert \le \Vert \mathbf{b}\Vert +\Vert \mathbf{c}\Vert <1+1=2.$

4. Thus, $\mathbf{a}\in 2B$.

Thus, $B+B=2B$.

Next, note that $B$ is symmetric; i.e. $B=-B$.

Thus,

$$B-B=B+(-B)=B+B=2B.$$

Theorem 4.38 (Moving a point into a ball)

Let $\mathbb{V}$ be a normed linear space. Let $B(\mathbf{x},r)$ be an open ball in $\mathbb{V}$.

Then, every point in $\mathbb{V}$ can be translated and scaled into a point which is inside the ball $B(\mathbf{x},r)$.

Proof. Let $\mathbf{v}\in \mathbb{V}$.

1. If $\mathbf{v}=\mathbf{0}$, then we can simply translate it by $\mathbf{x}$ to put it inside $B(\mathbf{x},r)$.

2. Now, assume that $\mathbf{v}\ne \mathbf{0}$. Then, $\Vert \mathbf{v}\Vert \ne 0$.

3. Let $\mathbf{y}=\mathbf{x}+\frac{r}{2\Vert \mathbf{v}\Vert }\mathbf{v}$.

4. Then,

   $$\Vert \mathbf{y}-\mathbf{x}\Vert =\Vert \frac{r}{2\Vert \mathbf{v}\Vert }\mathbf{v}\Vert =\frac{r}{2}<r.$$

5. Thus, $\mathbf{y}\in B(\mathbf{x},r)$.

In the following, $B$ means the open unit ball $B(\mathbf{0},1)$.

4.4.5. Interior

Theorem 4.39 (Interior points in a normed space)

Let $A$ be a subset of a normed linear space $\mathbb{V}$. Then, $\mathbf{x}\in \mathrm{int}A$ if and only if there exists $r>0$ such that

$$\mathbf{x}+rB\subseteq A.$$

Proof. From Definition 3.12, $\mathbf{x}$ is an interior point of $A$ if there exists an $r>0$ such that the open ball $B(\mathbf{x},r)\subseteq A$. But, as per algebraic notation $B(\mathbf{x},r)=\mathbf{x}+rB$.

Theorem 4.40 (Interior in a normed space)

Let $A$ be a subset of a normed linear space $\mathbb{V}$. The interior of $A$ is given by

$$\mathrm{int}A=\{\mathbf{x}|\mathrm{\exists }r>0,\mathbf{x}+rB\subseteq A\}.$$

Proof. This follows from the fact that the interior is a collection of all the interior points of $A$.

Theorem 4.41 (Subspaces and interior)

Let $\mathbb{V}$ be a normed vector space. The only subspace of $\mathbb{V}$ that has a nonempty interior is $\mathbb{V}$ itself.

Proof. Let $S$ be a subspace of $\mathbb{V}$.

1. Assume that $S$ has a nonempty interior.

2. Then, there is an interior point $\mathbf{x}\in S$.

3. Thus, there is an open ball $B(\mathbf{x},r)\subseteq S$.

4. By Theorem 4.38, any nonzero point $\mathbf{v}\in \mathbb{V}$ can be moved into the ball $B(\mathbf{x},r)$ as

   $$\mathbf{y}=\mathbf{x}+\frac{r}{2\Vert \mathbf{v}\Vert }\mathbf{v}.$$

5. We have $\mathbf{y}\in S$.

6. But a subspace is closed under vector addition and scalar multiplication.

7. Thus,

   $$\mathbf{v}=\frac{2\Vert \mathbf{v}\Vert }{r}(\mathbf{y}-\mathbf{x})\in S.$$

8. Thus, $\mathbf{v}\in \mathbb{V}$ implies $\mathbf{v}\in S$.

9. Thus, $\mathbb{V}=S$.

Thus, the only subspace which has a nonempty interior is $\mathbb{V}$ itself.

Corollary 4.10 (Interior of proper subspaces)

Every proper subspace of a normed linear space $\mathbb{V}$ has an empty interior.

4.4.6. Closure

Theorem 4.42 (Closure points in a normed space)

Let $A$ be a subset of a normed linear space $\mathbb{V}$. Then, $\mathbf{x}\in \mathrm{cl}A$ if and only if

$$\mathbf{x}\in A+rB\mathrm{\forall }r>0.$$

Proof. Recall that

$$A+rB=\bigcup _{\mathbf{a}\in A}\mathbf{a}+rB.$$

Thus, $\mathbf{x}\in A+rB$ if and only if there exists $\mathbf{a}\in A$ such that $\mathbf{x}\in \mathbf{a}+rB$.

Assume that $\mathbf{x}\in A+rB\mathrm{\forall }r>0$.

1. Thus, for every $r>0$, there exists $\mathbf{a}\in A$ (depending on $r$) such that $\mathbf{x}\in \mathbf{a}+rB=B(\mathbf{a},r)$.

2. Thus, for every $r>0$, there exists $\mathbf{a}\in A$ (depending on $r$) such that $d(\mathbf{x},\mathbf{a})<r$.

3. Thus, for every $r>0$, there exists $\mathbf{a}\in A$ (depending on $r$) such that $\mathbf{a}\in B(\mathbf{x},r)$.

4. Thus, $A\cap B(\mathbf{x},r)\ne \varnothing$ for every $r>0$.

5. Thus, $\mathbf{x}$ is a closure point of $A$.

For the converse, assume that $\mathbf{x}$ is a closure point of $A$.

1. Then, $A\cap B(\mathbf{x},r)\ne \varnothing$ for every $r>0$.

2. Thus, for every $r>0$, there exists $\mathbf{a}\in A$ (depending on $r$) such that $\mathbf{a}\in B(\mathbf{x},r)$.

3. Thus, for every $r>0$, there exists $\mathbf{a}\in A$ (depending on $r$) such that $d(\mathbf{x},\mathbf{a})<r$.

4. Thus, for every $r>0$, there exists $\mathbf{a}\in A$ (depending on $r$) such that $\mathbf{x}\in B(\mathbf{a},r)=\mathbf{a}+rB$.

5. Thus, for every $r>0$, $\mathbf{x}\in A+rB$.

Theorem 4.43 (Closure in a normed space)

Let $A$ be a subset of a normed linear space $\mathbb{V}$. Then, the closure of $A$ is given by

$$\mathrm{cl}A=\bigcap _{r>0}(A+rB).$$

Proof. From previous result, a point $\mathbf{x}$ is a closure point of $A$ if and only if

$$\mathbf{x}\in \bigcap _{r>0}(A+rB).$$

The result follows from the fact that the closure of $A$ is the collection of all its closure points.

4.4.7. Continuity

Definition 4.64 (Continuity in normed linear spaces)

Let $(\mathbb{V},\Vert \cdot {\Vert }_v)$ and $(\mathbb{W},\Vert \cdot {\Vert }_w)$ be normed linear spaces.

A function $f:\mathbb{V}\to \mathbb{W}$ between the two normed spaces is said to be continuous at a point $\mathbf{a}\in \mathrm{dom}f$ if for every $ϵ>0$, there exists $\delta >0$ (depending on $ϵ$ and $\mathbf{a}$) such that for all $\mathbf{x}\in \mathrm{dom}f$

$$\Vert \mathbf{x}-\mathbf{a}{\Vert }_v<\delta ⟹\Vert f(\mathbf{x})-f(\mathbf{a}){\Vert }_w<ϵ$$

holds true.

$f$ is said to be continuous on $A\subseteq \mathrm{dom}f$ if $f$ is continuous at every point of $A$.

The function $f$ is said to be uniformly continuous on $A\subseteq \mathrm{dom}f$ if for every $ϵ>0$, there exists some $\delta >0$ (depending on $ϵ$) such that

$$\Vert f(\mathbf{x})-f(\mathbf{y}){\Vert }_w<ϵ\text{ whenever }\Vert \mathbf{x}-\mathbf{y}{\Vert }_v<\delta \text{ and }\mathbf{x},\mathbf{y}\in A.$$

These definitions are just adapted from the corresponding definitions for metric spaces for convenience.

4.4.7.1. Norms

Theorem 4.44 (Norms are uniformly continuous )

A function $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$ satisfying all the properties of a norm is uniformly continuous in the metric space induced by the norm.

Proof. Let $\mathbf{x},\mathbf{y}\in \mathbb{V}$. Assume $d(\mathbf{x},\mathbf{y})<\delta$. Choose $ϵ=\delta$. Now, due to Theorem 4.33:

$$d(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}\Vert \ge |\Vert \mathbf{x}\Vert -\Vert \mathbf{y}\Vert |.$$

Thus,

$$d(\mathbf{x},\mathbf{y})<\delta ⟹|\Vert \mathbf{x}\Vert -\Vert \mathbf{y}\Vert |<\delta =ϵ.$$

Thus, $\Vert \cdot \Vert$ is uniformly continuous.

4.4.7.2. Translations

Theorem 4.45 (Translations are uniformly continuous )

Let $(\mathbb{V},\Vert \cdot \Vert )$ be a normed linear space. Let $\mathbf{a}\in \mathbb{V}$ be some fixed vector. Let a translation map $g_a:\mathbb{V}\to \mathbb{V}$ be defined as

$$g_a=\mathbf{x}+\mathbf{a}\mathrm{\forall }\mathbf{x}\in \mathbb{V}.$$

Then, $g_a$ is uniformly continuous.

Proof. Let $ϵ>0$ and $\delta =ϵ$. Let $\mathbf{x},\mathbf{y}\in \mathbb{V}$. Note that

$$\Vert g_a(\mathbf{x})-g_a(\mathbf{y})\Vert =\Vert \mathbf{x}+\mathbf{a}-(\mathbf{y}+\mathbf{a})\Vert =\Vert \mathbf{x}-\mathbf{y}\Vert .$$

Thus, $\Vert \mathbf{x}-\mathbf{y}\Vert <\delta$ implies $\Vert g_a(\mathbf{x})-g_a(\mathbf{y})\Vert <\delta =ϵ$.

Thus, $g_a$ is uniformly continuous.

Theorem 4.46 (Translations preserve topology)

Translations preserve the topology.

Let $(\mathbb{V},\Vert \cdot \Vert )$ be a normed linear space. Let $\mathbf{a}\in \mathbb{V}$ be some fixed vector. Let a translation map $g_a:\mathbb{V}\to \mathbb{V}$ be defined as

$$g_a=\mathbf{x}+\mathbf{a}\mathrm{\forall }\mathbf{x}\in \mathbb{V}.$$

Then,

1. $A$ is open if and only if $g_a(A)$ is open.

2. $A$ is closed if and only if $g_a(A)$ is closed.

3. $A$ is compact if and only if $g_a(A)$ is compact.

4. $A$ has an empty interior if and only if $g_a(A)$ has an empty interior.

Proof. We note that the inverse of $g_a$ is given by $g_{-a}$ which is also a translation. Consequently, both $g_a$ and $g_{-a}$ are uniformly continuous.

Preservation of open and closed sets are simple applications of Theorem 3.42 (characterization of continuous functions).

Preservation of compactness is an application of Theorem 3.78 (continuous images of compact sets are compact).

Assume that $A$ has an interior point $\mathbf{x}\in A$.

1. Then, there exists a ball $B(\mathbf{x},r)\subseteq A$.

2. Then, the ball $B(\mathbf{x},r)+\mathbf{a}=B(\mathbf{x}+\mathbf{a},r)\subseteq A+\mathbf{a}$.

3. Thus, $\mathbf{x}+ba$ is an interior point of $A+\mathbf{a}=g_a(A)$.

Thus, if $g_a(A)$ has an empty interior then $A$ must have an empty interior. The converse can be proved by using the inverse translation $g_{-a}$.

4.4.7.3. Scalar Multiplication

Theorem 4.47 (Scalar multiplication is uniformly continuous )

Let $(\mathbb{V},\Vert \cdot \Vert )$ be a normed linear space. Let $t\in \mathbb{F}$. Let a scalar multiplication map $g_t:\mathbb{V}\to \mathbb{V}$ be defined as

$$g_t=t\mathbf{x}\mathrm{\forall }\mathbf{x}\in \mathbb{V}.$$

Then, $g_t$ is uniformly continuous.

Proof. If $t=0$, then $g_t(\mathbf{x})=\mathbf{0}$ for all $\mathbf{x}\in \mathbb{V}$ so it is trivially uniformly continuous.

Assume $t\ne 0$.

1. Let $ϵ>0$.

2. Choose $\delta =\frac{ϵ}{|t|}$.

3. Now, for any $\mathbf{x},\mathbf{y}\in \mathbb{V}$

   $$\Vert g_t(\mathbf{x})-g_t(\mathbf{y})\Vert =\Vert t\mathbf{x}-t\mathbf{y}\Vert =\Vert t(\mathbf{x}-\mathbf{y})\Vert =|t|\Vert \mathbf{x}-\mathbf{y}\Vert .$$

4. Thus, $\Vert \mathbf{x}-\mathbf{y}\Vert <\delta$ implies

   $$\Vert g_t(\mathbf{x})-g_t(\mathbf{y})\Vert <|t|\delta =ϵ.$$

5. Thus, $g_t$ is uniformly continuous.

Theorem 4.48 (Scalar multiplication preserves topology )

Let $(\mathbb{V},\Vert \cdot \Vert )$ be a normed linear space. Let $t\in \mathbb{F}$ be nonzero (i.e., $t\ne 0$). Let a scalar multiplication map $g_t:\mathbb{V}\to \mathbb{V}$ be defined as

$$g_t=t\mathbf{x}\mathrm{\forall }\mathbf{x}\in \mathbb{V}.$$

Then, $g_t$ is a homeomorphism. Consequently,

1. $A$ is open if and only if $g_t(A)$ is open.

2. $A$ is closed if and only if $g_t(A)$ is closed.

3. $A$ is compact if and only if $g_t(A)$ is compact.

4. $A$ has an empty interior if and only if $g_t(A)$ has an empty interior.

Proof. We first show that $g_t$ is a homeomorphism. Since $t\ne 0$, let $r=\frac{1}{t}$.

1. $g_t$ is uniformly continuous by Theorem 4.47.

2. Then, $g_r$ is also uniformly continuous by Theorem 4.47.

3. Note that

   $$g_r(g_t)(\mathbf{x})=g_t(g_r(\mathbf{x}))=\mathbf{x}\mathrm{\forall }\mathbf{x}\in \mathbb{V}.$$

4. Thus, $g_t$ is bijective and $g_r$ is its inverse.

5. Since $g_t$ is bijective, $g_t$ is continuous, and its inverse $g_r$ is continuous, hence $g_t$ is a homeomorphism.

6. Morever, $g_r$ is also a homeomorphism.

By Theorem 3.49, $g_t$ and $g_r$ are both a closed mapping (mapping closed sets to closed sets) and an open mapping (mapping open sets to open sets). Consequently,

1. $A$ is open if and only if $g_t(A)$ is open.

2. $A$ is closed if and only if $g_t(A)$ is closed.

Due to Theorem 3.80 (homeomorphisms preserve compact sets) $A$ is compact if and only if $g_t(A)$ is compact.

Due to Theorem 3.52, homeomorphisms preserve interiors. Hence, $A$ has an empty interior if and only if $g_t(A)$ has an empty interior.

4.4.8. Open Sets

Theorem 4.49 (Sum of an open set with any set is open)

Let $\mathbb{V}$ be a normed linear space. Let $A,B\subseteq \mathbb{V}$. If $A$ is open, then the sum $A+B$ is open.

Proof. We proceed as follows:

1. Let $x\in B$.

2. Then the set $x+A$ is open since $A$ is open and translations preserve open sets (Theorem 4.46).

3. Then,

   $$A+B=\bigcup _{x\in B}x+A$$

   is a union of open sets. Hence, it is open.

4.4.9. Closed Sets

Theorem 4.50

If $A$ is closed and $B$ is compact, then their sum $A+B$ is closed.

Proof. Let $\{z_n\}$ with $z_n=a_n+b_n\in A+B$ be a convergent sequence of $A+B$ where $a_n\in A$ and $b_n\in B$.

1. $\{a_n\}$ is a sequence of $A$.

2. $\{b_n\}$ is a sequence of $B$.

3. Let $lim{z}_n=z$.

4. Since $B$ is compact, $\{b_n\}$ has a convergent subsequence, say $\{b_{k_n}\}$ (Theorem 3.75).

5. Let $lim{b}_{k_n}=l\in B$.

6. Now consider the sequence $\{a_{k_n}\}$ given by $a_{k_n}=z_{k_n}-b_{k_n}$.

7. $\{z_{k_n}\}$ is convergent since it is a subsequence of a convergent sequence (Theorem 3.35).

8. Since $\{z_{k_n}\}$ and $\{b_{k_n}\}$ are both convergent, hence $\{a_{k_n}\}$ is also convergent.

9. Let $lim{a}_{k_n}=m$.

10. Since $A$ is closed. Hence $m\in A$ (Theorem 3.33).

11. Now, $lim{a}_{k_n}=lim{z}_{k_n}-lim{b}_{k_n}$ gives us $m=z-l$.

12. Thus, $z=m+l$.

13. But $m\in A$ and $l\in B$.

14. Hence, $z\in A+B$.

15. Thus, every convergent sequence of $A+B$ converges in $A+B$.

16. Thus, $A+B$ is closed. (Theorem 3.33).

4.4.10. Boundedness

By definition $\Vert \mathbf{x}\Vert \ge 0$. Thus, for any subset $A\subseteq \mathbb{V}$, $0$ is a lower bound for the set $\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}$. If there is an upper bound also for the set of norms, then the set $A$ is called bounded.

Definition 4.65 (Bounded set)

Let $\mathbb{V}$ be a normed linear space. A subset $A$ of $\mathbb{V}$ is called norm bounded or simply bounded if there exist $M>0$ such that:

$$\Vert \mathbf{x}\Vert \le M\mathrm{\forall }\mathbf{x}\in A.$$

Compare the definition with the definition of bounded sets in metric spaces.

We can characterize the case where $0$ is indeed the infimum or greatest lower bound for the set $\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}$.

Theorem 4.51 (Zero as the greatest lower bound)

Let $\mathbb{V}$ be a normed linear space. Let $A\subseteq \mathbb{V}$. Then, $inf\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}=0$ if and only if $\mathbf{0}\in \mathrm{cl}A$.

Proof. Note that $B(\mathbf{0},r)$ is an open ball of radius $r$ around $\mathbf{0}$ given by:

$$B(\mathbf{0},r)=\{\mathbf{x}\in \mathbb{V}|\Vert \mathbf{x}\Vert <r\}.$$

Suppose $inf\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}=0$.

1. Then, for every $r>0$, there exists $\mathbf{x}\in A$ such that $\Vert \mathbf{x}\Vert <r$.

2. Thus, for every $r>0$, $\mathbf{x}\in B(\mathbf{0},r)\cap A$.

3. Thus, for every $r>0$, $B(\mathbf{0},r)\cap A\ne \varnothing$.

4. Thus, $\mathbf{0}\in \mathrm{cl}A$.

Now suppose that $\mathbf{0}\in \mathrm{cl}A$.

1. Then, for every $r>0$, $B(\mathbf{0},r)\cap A\ne \varnothing$.

2. Thus, for every $r>0$, there exists $\mathbf{x}\in A$ such that $\Vert \mathbf{x}\Vert <r$.

3. Thus, $inf\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}=0$.

4.4.11. Sequences

Definition 4.66 (Convergence in norm)

A sequence $\{{\mathbf{x}}_n\}$ of a normed space $\mathbb{V}$ is said to converge in norm to $\mathbf{x}\in \mathbb{V}$ if

$$\underset{n\to \mathrm{\infty }}{lim}\Vert \mathbf{x}-{\mathbf{x}}_n\Vert =0;$$

i.e., if $\{{\mathbf{x}}_n\}$ converges to $\mathbf{x}$ with respect to the metric induced by the norm. We write this as:

$$\underset{n\to \mathrm{\infty }}{lim}{\mathbf{x}}_n=\mathbf{x}.$$

Following is an alternative proof for Theorem 4.51 in terms of convergent sequences.

Proof. Assume that $\mathbf{0}\in \mathrm{cl}A$.

1. Then, there exists a sequence $\{{\mathbf{x}}_n\}$ such that $lim\Vert {\mathbf{x}}_n-\mathbf{0}\Vert =0$.

2. Thus, $lim\Vert {\mathbf{x}}_n\Vert =0$.

3. Thus, for every $r>0$, there exists $n_0\in \mathbb{N}$ such that $\Vert {\mathbf{x}}_n\Vert <r$ for all $n>n_0$.

4. Thus, for every $r>0$, there exists $\mathbf{x}\in A$ such that $\Vert \mathbf{x}\Vert <r$.

5. Thus, $inf\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}=0$.

Assume that $inf\{\Vert \mathbf{x}\Vert |\mathbf{x}\in A\}=0$.

1. Thus, for every $r>0$, there exists $\mathbf{x}\in A$ such that $\Vert \mathbf{x}\Vert <r$.

2. For every $n\in \mathbb{N}$, pick a ${\mathbf{x}}_n\in A$ such that $\Vert {\mathbf{x}}_n\Vert <\frac{1}{n}$.

3. Form the sequence $\{{\mathbf{x}}_n\}$.

4. Then, for every $r>0$, there exists $n_0\in \mathbb{N}$ such that for all $n>n_0$, $\Vert {\mathbf{x}}_n\Vert =\Vert {\mathbf{x}}_n-\mathbf{0}\Vert <r$.

5. Thus, $lim{\mathbf{x}}_n=\mathbf{0}$.

6. Thus, $\mathbf{0}\in \mathrm{cl}A$.

4.4.12. The Calculus of Limits

Let $\{{\mathbf{x}}_n\}$ and $\{{\mathbf{y}}_n\}$ be convergent sequences of $\mathbb{V}$. Our concern here is to understand what happens to the limits if the sequences are combined.

Our presentation here is similar to the presentation for sequences of real numbers in The Calculus of Limits.

Let $lim\{{\mathbf{x}}_n\}=\mathbf{x}$ and $lim\{{\mathbf{y}}_n\}=\mathbf{y}$. Then:

Theorem 4.52 (Scaling a sequence)

$$lim\{\alpha {\mathbf{x}}_n\}=\alpha \mathbf{x}\mathrm{\forall }\alpha \in \mathbb{F}.$$

Proof. If $\alpha =0$, then we have a constant sequence and the result is trivial. So assume that $\alpha \ne 0$. Then:

$$\Vert \alpha {\mathbf{x}}_n-\alpha \mathbf{x}\Vert =|\alpha |\Vert {\mathbf{x}}_n-\mathbf{x}\Vert .$$

Let $ϵ>0$ and choose $n_0\in \mathbb{N}$ such that $\Vert \mathbf{x}-{\mathbf{x}}_n\Vert <\frac{ϵ}{|\alpha |}$ for all $n>n_0$. Then

$$\Vert \alpha {\mathbf{x}}_n-\alpha \mathbf{x}\Vert =|\alpha |\Vert {\mathbf{x}}_n-\mathbf{x}\Vert <|\alpha |\frac{ϵ}{|\alpha |}=ϵ\mathrm{\forall }n>n_0.$$

Corollary 4.11 (Negating a sequence)

$$lim\{-{\mathbf{x}}_n\}=-\mathbf{x}.$$

We get this result by choosing $\alpha =-1$.

Theorem 4.53 (Addition of sequences)

$$lim\{{\mathbf{x}}_n+{\mathbf{y}}_n\}=\mathbf{x}+\mathbf{y}.$$

Proof. From triangle inequality we get:

$$\Vert {\mathbf{x}}_n+{\mathbf{y}}_n-(\mathbf{x}+\mathbf{y})|\le \Vert {\mathbf{x}}_n-\mathbf{x}\Vert +\Vert {\mathbf{y}}_n-\mathbf{y}\Vert .$$

For any $ϵ>0$, choose $n_1$ such that $\Vert {\mathbf{x}}_n-\mathbf{x}\Vert <\frac{ϵ}{2}\mathrm{\forall }n>n_1$.

Similarly, choose $n_2$ such that $\Vert {\mathbf{y}}_n-\mathbf{y}\Vert <\frac{ϵ}{2}\mathrm{\forall }n>n_2$.

Now choose $n_0=max(n_1,n_2)$. Then:

$$\Vert {\mathbf{x}}_n+{\mathbf{y}}_n-(\mathbf{x}+\mathbf{y})\Vert \le \Vert {\mathbf{x}}_n-\mathbf{x}\Vert +\Vert {\mathbf{y}}_n-\mathbf{y}\Vert <\frac{ϵ}{2}+\frac{ϵ}{2}=ϵ\mathrm{\forall }n>n_0.$$

Corollary 4.12 (Subtraction of sequences)

$$lim\{{\mathbf{x}}_n-{\mathbf{y}}_n\}=\mathbf{x}-\mathbf{y}.$$

Negate $\{{\mathbf{y}}_n\}$ and add to $\{{\mathbf{x}}_n\}$.

4.4.13. Cauchy Sequences

Theorem 4.54 (Cauchy sequence is bounded)

Every Cauchy sequence in a normed space is bounded.

The proof is very similar to the proof for the boundedness of Cauchy sequences in real line (Theorem 2.18).

Proof. Let $\{{\mathbf{x}}_n\}$ be a sequence of a normed space $\mathbb{V}$.

1. Choose $ϵ=1$.

2. Then there exists $n_0\in \mathbb{N}$ such that $\Vert {\mathbf{x}}_n-{\mathbf{x}}_m\Vert <1$ whenever $m,n\ge n_0$.

3. In particular, the statement is valid when $m=n_0$. i.e. $\Vert {\mathbf{x}}_n-{\mathbf{x}}_{n_0}\Vert <1$.

4. But,

   $$\begin{array}{r}\begin{array}{rl}& \Vert {\mathbf{x}}_n-{\mathbf{x}}_{n_0}\Vert <1\\ & ⟹|\Vert {\mathbf{x}}_n\Vert -\Vert {\mathbf{x}}_{n_0}\Vert |<1\\ & ⟹\Vert {\mathbf{x}}_n\Vert <1+\Vert {\mathbf{x}}_{n_0}\Vert \mathrm{\forall }n\ge n_0.\end{array}\end{array}$$

5. Choosing $M=max(\Vert x_1\Vert ,\dots ,\Vert {\mathbf{x}}_{n_0-1}\Vert ,\Vert {\mathbf{x}}_{n_0}\Vert +1)$, it is clear that $\Vert x_n\Vert \le M$

6. Hence $\{{\mathbf{x}}_n\}$ is bounded.

4.4.14. Linear Transformations

4.4.14.1. Boundedness

Definition 4.67 (Bounded linear transformation)

Let $(\mathbb{V},\Vert \cdot {\Vert }_v)$ and $(\mathbb{W},\Vert \cdot {\Vert }_w)$ be normed linear spaces. Let $T:\mathbb{V}\to \mathbb{W}$ be a linear transformation. We say that $T$ is bounded (in the sense of linear transformations) if the set

(4.1)$$S\triangleq \{\frac{\Vert T(\mathbf{x}){\Vert }_w}{\Vert \mathbf{x}{\Vert }_v}|\mathbf{x}\in \mathbb{V},\mathbf{x}\ne \mathbf{0}\}$$

is bounded.

Note that the set $S$ in (4.1) is trivially bounded from below by $0$. Thus, by bounded we mean, bounded from above.

The notion of boundedness for linear transformations is different from the notion of boundedness for bounded functions in general metric spaces as defined in Definition 3.37. There, we posit that the range of the function is bounded. For bounded linear transformations, the range (in norm) may be unbounded, yet the ratio of the norms of output and input is bounded.

Since linear transformations commute with scalar multiplication as in $T(t\mathbf{x})=tT(\mathbf{x})$, hence the norm of the output is unbounded unless $T$ is an identically $\mathbf{0}$ linear transformation.

If there exists $\mathbf{v}\in \mathbb{V}$ such that $T(\mathbf{v})\ne \mathbf{0}$, then

$$\Vert T(t\mathbf{v}){\Vert }_w=|t|\Vert T(\mathbf{v}){\Vert }_w\to \mathrm{\infty }\text{ as }|t|\to \mathrm{\infty }.$$

Thus, the idea of a bounded range is not very useful for linear transformations.

Remark 4.8 (Bounded linear transformation in terms of unit norm vectors)

For $T,\mathbf{x}$ as in (4.1), and any nonzero $t\in \mathbb{F}$, we have

$$\frac{\Vert T(t\mathbf{x}){\Vert }_w}{\Vert t\mathbf{x}{\Vert }_v}=\frac{\Vert tT(\mathbf{x}){\Vert }_w}{|t|\Vert \mathbf{x}{\Vert }_v}=\frac{|t|\Vert T(\mathbf{x}){\Vert }_w}{|t|\Vert \mathbf{x}{\Vert }_v}=\frac{\Vert T(\mathbf{x}){\Vert }_w}{||\Vert \mathbf{x}{\Vert }_v}.$$

In particular, if we choose $t=\frac{1}{\Vert \mathbf{x}{\Vert }_v}$, then $\Vert t\mathbf{x}{\Vert }_v=1$.

Hence, every element in set $S$ in (4.1) equals $\Vert T(\mathbf{x}){\Vert }_w$ for some unit vector $\mathbf{x}$. It follows that

(4.2)$$S=\{\frac{\Vert T(\mathbf{x}){\Vert }_w}{\Vert \mathbf{x}{\Vert }_v}|\mathbf{x}\in \mathbb{V},\mathbf{x}\ne \mathbf{0}\}=\{\Vert T(\mathbf{x}){\Vert }_w|\mathbf{x}\in \mathbb{V},\Vert \mathbf{x}{\Vert }_v=1\}.$$

Some authors prefer this description of $S$ as the definition of bounded linear transformations.

Example 4.18 (Unbounded linear transformation)

It is possible to construct unbounded linear transformations in infinite dimensional normed linear spaces.

We shall consider a sequence space as described below.

Let $V$ be the set of all real sequences with finitely many nonzero terms.

$$V=\{\{x_n\}|\mathrm{\exists }m\in \mathbb{N}\text{ with }{x}_n=0\mathrm{\forall }n\ge m\}.$$

It is easy to verify that $V$ is a linear subspace of the space of all sequences ${\mathbb{R}}^{\mathrm{\infty }}$.

For any $\mathbf{x}\in V$ given by $\mathbf{x}=\{x_n\}$, define the supremum norm as:

$$\Vert \mathbf{x}\Vert =sup\{|x_1|,|x_2|,\dots \}.$$

It is easy to verify that it is indeed a norm.

Define a transformation $T:V\to V$ as

$$T(x_1,x_2,x_3,\dots ,x_n,0,0,\dots )=(x_1,2x_2,3x_3,\dots ,n{x}_n,0,0,\dots ).$$

It is easy to verify that $T$ is linear.

For any sequence ${\mathbf{e}}_n=(0,0,\dots ,1,0,0,\dots )$ with $1$ in n-th position and all other entries 0, $\Vert {\mathbf{e}}_n\Vert =1$ and $\Vert T({\mathbf{e}}_n)\Vert =n$. Since we can construct ${\mathbf{e}}_n$ for every $n\in \mathbb{N}$, hence the set $S$ as defined in (4.2) is unbounded.

Consequently, $T$ is an unbounded linear transformation.

4.4.14.2. Continuity

Theorem 4.55 (Characterization of continuity for linear transformations)

Let $(\mathbb{V},\Vert \cdot {\Vert }_v)$ and $(\mathbb{W},\Vert \cdot {\Vert }_w)$ be normed linear spaces. Let $T:\mathbb{V}\to \mathbb{W}$ be a linear transformation. Then, the following statements are equivalent.

1. $T$ is continuous at one point in $\mathbb{V}$.

2. $T$ is continuous at $\mathbf{0}\in \mathbb{V}$.

3. $T$ is bounded.

4. $T$ is Lipschitz continuous at $\mathbf{0}\in \mathbb{V}$.

5. $T$ is Lipschitz continuous.

6. $T$ is uniformly continuous.

7. $T$ is continuous.

Proof. (1) $⟹$ (2)

Assume $T$ is continuous at ${\mathbf{x}}_0\in \mathbb{V}$.

1. Let $ϵ>0$ and let $\delta >0$ such that if $\Vert \mathbf{x}-{\mathbf{x}}_0{\Vert }_v<\delta$ then $\Vert T(\mathbf{x})-T({\mathbf{x}}_0){\Vert }_w<ϵ$.

2. Then, for all $\mathbf{y}\in B_v(\mathbf{0},\delta )$, we have

   $$\Vert (\mathbf{y}+{\mathbf{x}}_0)-{\mathbf{x}}_0{\Vert }_v=\Vert \mathbf{y}{\Vert }_v<\delta .$$

3. Hence, $\Vert T(\mathbf{y}+{\mathbf{x}}_0)-T({\mathbf{x}}_0){\Vert }_w<ϵ$.

4. But $T$ is linear. Hence

   $$T(\mathbf{y}+{\mathbf{x}}_0)-T({\mathbf{x}}_0)=T(\mathbf{y})=T(\mathbf{y})-\mathbf{0}=T(\mathbf{y})-T(\mathbf{0}).$$

5. Thus, $\Vert T(\mathbf{y})-T(\mathbf{0}){\Vert }_w<ϵ$.

6. Thus, for any $ϵ>0$, there exists $\delta >0$ such that $\Vert \mathbf{y}-\mathbf{0}{\Vert }_v<\delta$ implies $\Vert T(\mathbf{y})-T(\mathbf{0}){\Vert }_w<ϵ$.

7. Thus, $T$ is continuous at $\mathbf{0}\in \mathbb{V}$.

(2) $⟹$ (3)

Assume that $T$ is continuous at $\mathbf{0}\in \mathbb{V}$.

1. Then, for any $\mathbf{y}\in \mathbb{V}$,

   $$\Vert T(\mathbf{y})-T(\mathbf{0}){\Vert }_w=\Vert T(\mathbf{y})-\mathbf{0}{\Vert }_w=\Vert T(\mathbf{y}){\Vert }_w.$$

2. Let $ϵ=1$ and choose $\delta >0$ such that if $\Vert \mathbf{y}{\Vert }_v<\delta$ then $\Vert T(\mathbf{y}){\Vert }_w<1$. We can choose such $\delta >0$ since $T$ is continuous at $\mathbf{0}$.

3. Let $\mathbf{x}\in \mathbb{V}$ be any nonzero vector.

4. Let $\mathbf{y}=\frac{\delta }{2\Vert \mathbf{x}{\Vert }_v}\mathbf{x}$.

5. Then, $\Vert \mathbf{y}{\Vert }_v=\frac{\delta }{2}<\delta$.

6. And $\mathbf{x}=\frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }\mathbf{y}$.

7. Then,

   $$\begin{array}{rl}\Vert T(\mathbf{x}){\Vert }_w& ={\Vert T\left(\frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }\mathbf{y}\right)\Vert }_w\\ & ={\Vert \frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }T(\mathbf{y})\Vert }_w\\ & =\frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }\Vert T(\mathbf{y}){\Vert }_w\\ & <\frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }\times 1\\ & =\frac{2}{\delta }\Vert \mathbf{x}{\Vert }_v.\end{array}$$

8. Thus,

   $$\frac{\Vert T(\mathbf{x}){\Vert }_w}{\Vert \mathbf{x}{\Vert }_v}<\frac{2}{\delta }$$

   holds true for every nonzero $\mathbf{x}\in \mathbb{V}$.

9. Thus, the set in (4.1) is bounded from above by $\frac{2}{\delta }$.

10. Thus, $T$ is bounded.

(3) $⟹$ (4)

Assume that $T$ is bounded. Let the set $S$ in (4.1) have an upper bound $M>0$. Let $\mathbf{x}\in \mathbb{V}$ be a nonzero vector. Then,

$$\begin{array}{rl}\Vert T(\mathbf{x})-T(\mathbf{0}){\Vert }_w& =\Vert T(\mathbf{x})-\mathbf{0}{\Vert }_w\\ & =\Vert T(\mathbf{x}){\Vert }_w\\ & ={\Vert T(\Vert \mathbf{x}{\Vert }_v\frac{\mathbf{x}}{\Vert \mathbf{x}{\Vert }_v})\Vert }_w\\ & =\Vert \mathbf{x}{\Vert }_v{\Vert T\left(\frac{\mathbf{x}}{\Vert \mathbf{x}{\Vert }_v}\right)\Vert }_w\\ & \le \Vert \mathbf{x}{\Vert }_{v}M\\ & =M\Vert \mathbf{x}-\mathbf{0}{\Vert }_v.\end{array}$$

Thus, there exists $M>0$ such that for every nonzero $\mathbf{v}\in \mathbb{V}$,

$$\Vert T(\mathbf{x})-T(\mathbf{0}){\Vert }_w\le M\Vert \mathbf{x}-\mathbf{0}{\Vert }_v.$$

Thus, $T$ is Lipschitz continuous at $\mathbf{0}\in \mathbb{V}$.

(4) $⟹$ (5)

We assume that $T$ is Lipschitz continuous at $\mathbf{0}\in \mathbb{V}$.

Then, let $K,\delta >0$ such that for every $\mathbf{y}\in \mathbb{V}$ with $\Vert \mathbf{y}{\Vert }_v<\delta$, we have

$$\Vert T(\mathbf{y}){\Vert }_w\le K\Vert \mathbf{y}{\Vert }_v.$$

1. Let $\mathbf{x}\in \mathbb{V}$ be a nonzero vector.

2. Let $\mathbf{y}=\frac{\delta }{2}\frac{\mathbf{x}}{\Vert \mathbf{x}{\Vert }_v}$.

3. Then, $\Vert \mathbf{y}{\Vert }_v=\frac{\delta }{2}<\delta$.

4. Hence, $\Vert T(\mathbf{y}){\Vert }_w\le K\Vert \mathbf{y}{\Vert }_v$.

5. Now, using linearity of $T$:

   $$\begin{array}{rl}\Vert T(\mathbf{x}){\Vert }_w& ={\Vert T\left(\frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }\mathbf{y}\right)\Vert }_w\\ & ={\Vert \frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }T(\mathbf{y})\Vert }_w\\ & =\frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }\Vert T(\mathbf{y}){\Vert }_w\\ & \le \frac{2\Vert \mathbf{x}{\Vert }_v}{\delta }K\Vert \mathbf{y}{\Vert }_v\\ & =K\Vert \mathbf{x}{\Vert }_v.\end{array}$$

6. Thus, we have $\Vert T(\mathbf{x}){\Vert }_w\le K\Vert \mathbf{x}{\Vert }_v$ for every nonzero $\mathbf{x}\in \mathbb{V}$.

7. Now, let $\mathbf{x},\mathbf{z}\in \mathbb{V}$ be distinct.

8. Using linearity of $T$

   $$\begin{array}{rl}\Vert T(\mathbf{x})-T(\mathbf{z}){\Vert }_w& =\Vert T(\mathbf{x}-\mathbf{z}){\Vert }_w\\ \le K\Vert \mathbf{x}-\mathbf{z}{\Vert }_v.\end{array}$$

9. Hence, $T$ is Lipschitz continuous.

(5) $⟹$ (6)

Assume that $T$ is Lipschitz continuous.

Let $K>0$ such that for every $\mathbf{x},\mathbf{y}\in \mathbb{V}$,

$$\Vert T(\mathbf{x})-T(\mathbf{y}){\Vert }_w\le K\Vert \mathbf{x}-\mathbf{y}{\Vert }_v.$$

1. Let $ϵ>0$.

2. Choose $\delta =\frac{ϵ}{K}$.

3. Then, for every $\mathbf{x},\mathbf{y}\in \mathbb{V}$ with $\Vert \mathbf{x}-\mathbf{y}{\Vert }_v<\delta$, we have

   $$\Vert T(\mathbf{x})-T(\mathbf{y}){\Vert }_w\le K\Vert \mathbf{x}-\mathbf{y}{\Vert }_v<K\delta =K\frac{ϵ}{K}=ϵ.$$

4. Hence, $T$ is uniformly continuous.

(6) $⟹$ (7)

A uniformly continuous function is trivially continuous.

(7) $⟹$ (1)

A continuous function is trivially continuous at a point.

4.4.15. Equivalent Norms

Definition 4.68 (Equivalent norms)

Let $\mathbb{V}$ be a vector space. Let $\Vert \cdot {\Vert }_a:\mathbb{V}\to \mathbb{R}$ and $\Vert \cdot {\Vert }_b:\mathbb{V}\to \mathbb{R}$ be two different norms defined on $\mathbb{V}$.

We say that the two norms are equivalent if there exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

$$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

Note that the zero-dimensional vector space $\{\mathbf{0}\}$ has only one norm which is identically zero. Thus, the concept of equivalent norms is not interesting for it. Thus, we shall concern ourselves only with vector spaces of dimension greater than zero for the purposes of norm equivalence.

There are several ways to express the equivalent norm inequalities.

Theorem 4.56 (Characterization of equivalent norms)

Let $\mathbb{V}$ be a vector space. Let $\Vert \cdot {\Vert }_a:\mathbb{V}\to \mathbb{R}$ and $\Vert \cdot {\Vert }_b:\mathbb{V}\to \mathbb{R}$ be two different norms defined on $\mathbb{V}$.

The following statements are equivalent.

1. The norms $\Vert \cdot {\Vert }_a$ and $\Vert \cdot {\Vert }_b$ are equivalent.

2. There exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$

   $$c_1\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a\le c_2\Vert \mathbf{v}{\Vert }_b.$$

3. There exists $c>0$ such that for all $\mathbf{v}\in \mathbb{V}$

   $$\frac{1}{c}\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a\le c\Vert \mathbf{v}{\Vert }_b.$$

Proof. (1) $⟹$ (2)

1. By hypothesis, the norms $\Vert \cdot {\Vert }_a$ and $\Vert \cdot {\Vert }_b$ are equivalent.

2. Thus, there exist $d_1,d_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$

   $$\Vert \mathbf{v}{\Vert }_a\le d_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le d_2\Vert \mathbf{v}{\Vert }_a.$$

3. Let $c_1=\frac{1}{d_2}>0$ and $c_2=d_1>0$.

4. The inequalities become:

   $$\Vert \mathbf{v}{\Vert }_a\le c_2\Vert \mathbf{v}{\Vert }_b\text{ and }{c}_1\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a.$$

5. Combining, we get:

   $$c_1\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a\le c_2\Vert \mathbf{v}{\Vert }_b.$$

(2) $⟹$ (3)

1. By hypothesis, there exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$

   $$c_1\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a\le c_2\Vert \mathbf{v}{\Vert }_b.$$

2. Let $c=max\{\frac{1}{c_1},c_2\}$. Note that $c>0$ since $c_1,c_2>0$.

3. Then, $c\ge c_2$.

4. Thus, $\Vert \mathbf{v}{\Vert }_a\le c_2\Vert \mathbf{v}{\Vert }_b\le c\Vert \mathbf{v}{\Vert }_b$ for every $\mathbf{v}\in \mathbb{V}$.

5. Also, $c\ge \frac{1}{c_1}⟹\frac{1}{c}\le c_1$.

6. Thus, $\frac{1}{c}\Vert \mathbf{v}{\Vert }_b\le c_1\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a$ for every $\mathbf{v}\in \mathbb{V}$.

7. Combining, we get:

   $$\frac{1}{c}\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a\le c\Vert \mathbf{v}{\Vert }_b.$$

(3) $⟹$ (1)

1. By hypothesis, there exists $c>0$ such that for all $\mathbf{v}\in \mathbb{V}$

   $$\frac{1}{c}\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a\le c\Vert \mathbf{v}{\Vert }_b.$$

2. Let $c_1=c$ and $c_2=c$.

3. Thus, $\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b$ for all $\mathbf{v}\in \mathbb{V}$.

4. And $\frac{1}{c_2}\Vert \mathbf{v}{\Vert }_b\le \Vert \mathbf{v}{\Vert }_a$ implies $\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a$ for all $\mathbf{v}\in \mathbb{V}$.

5. Thus, there exist constants $c_1,c_2>0$ given by $c_1=c_2=c$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

   $$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

6. Thus, the two norms are equivalent.

Theorem 4.57 (Norm equivalence = Identical bounded sets)

Let $\mathbb{V}$ be a vector space. Let $\Vert \cdot {\Vert }_a:\mathbb{V}\to \mathbb{R}$ and $\Vert \cdot {\Vert }_b:\mathbb{V}\to \mathbb{R}$ be two different norms defined on $\mathbb{V}$.

The following statements are equivalent.

1. The norms $\Vert \cdot {\Vert }_a$ and $\Vert \cdot {\Vert }_b$ are equivalent.

2. Every set $A\subseteq \mathbb{V}$ is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_a)$ if and only if it is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_b)$.

Proof. (1) $⟹$ (2)

1. By hypothesis, the norms $\Vert \cdot {\Vert }_a$ and $\Vert \cdot {\Vert }_b$ are equivalent.

2. Thus, there exist $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$

   $$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

3. Let $A$ be bounded in $(\mathbb{V},\Vert \cdot {\Vert }_a)$.

4. Then, there exists a constant $M>0$ such that

   $$\Vert \mathbf{x}{\Vert }_a\le M\mathrm{\forall }\mathbf{x}\in A.$$

5. But then,

   $$\Vert \mathbf{x}{\Vert }_b\le c_2\Vert \mathbf{x}{\Vert }_a\le M{c}_2\mathrm{\forall }\mathbf{x}\in A.$$

6. Thus, $A$ is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_b)$.

7. A similar reasoning shows that if $A$ is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_b)$, then $A$ must be bounded in $(\mathbb{V},\Vert \cdot {\Vert }_a)$ also.

(2) $⟹$ (1)

1. By hypothesis, every set $A\subseteq \mathbb{V}$ is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_a)$ if and only if it is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_b)$.

2. For contradiction, assume that the norms are not equivalent and there is no constant $c_1>0$ such that $\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b$ for every $\mathbf{v}\in \mathbb{V}$.

3. Then, for each $n\in \mathbb{N}$, there exists ${\mathbf{x}}_n\in \mathbb{V}$ such that $\Vert {\mathbf{x}}_n{\Vert }_a>n\Vert {\mathbf{x}}_n{\Vert }_b$.

4. In particular, ${\mathbf{x}}_n\ne \mathbf{0}$.

5. The set $S=\{\frac{{\mathbf{x}}_n}{\Vert {\mathbf{x}}_n{\Vert }_b}|n\in \mathbb{N}\}$ is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_b)$ since each point is unit norm.

6. Then, $S$ is bounded in $(\mathbb{V},\Vert \cdot {\Vert }_a)$ also by hypothesis.

7. Thus, ${\Vert \frac{{\mathbf{x}}_n}{\Vert {\mathbf{x}}_n{\Vert }_b}\Vert }_a\le C$ for some $C>0$ and every $n\in \mathbb{N}$.

8. It implies $\Vert {\mathbf{x}}_n{\Vert }_a\le C\Vert {\mathbf{x}}_n{\Vert }_b$ for every $n\in \mathbb{N}$.

9. But, this contradictions with our choice above as $\Vert {\mathbf{x}}_n{\Vert }_a>n\Vert {\mathbf{x}}_n{\Vert }_b$.

10. Thus, there must exist a constant $c_1>0$ such that $\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b$ for every $\mathbf{v}\in \mathbb{V}$.

11. A similar reasoning shows that there must exist a constant $c_2>0$ such that $\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a$ for every $\mathbf{v}\in \mathbb{V}$.

12. Thus, the two norms must be equivalent.

Theorem 4.58

Equivalence of norms is an equivalence relation on the set of norms for a given vector space.

Proof. Let $\mathbb{V}$ be a given vector space. Let $\Vert \cdot {\Vert }_a:\mathbb{V}\to \mathbb{R}$, $\Vert \cdot {\Vert }_b:\mathbb{V}\to \mathbb{R}$, $\Vert \cdot {\Vert }_c:\mathbb{V}\to \mathbb{R}$, be norms defined on $\mathbb{V}$.

We shall say that $\Vert \cdot {\Vert }_a\sim \Vert \cdot {\Vert }_b$ if the norms $\Vert \cdot {\Vert }_a$ and $\Vert \cdot {\Vert }_b$ are equivalent; i.e., there exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

$$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

[Reflexivity]

1. Choose $c_1=c_2=1$. Then,

   $$\Vert \mathbf{v}{\Vert }_a\le \Vert \mathbf{v}{\Vert }_a$$

   for every $\mathbf{v}\in \mathbb{V}$.

2. Hence, $\Vert \cdot {\Vert }_a\sim \Vert \cdot {\Vert }_a$.

[Symmetry]

1. Let $\Vert \cdot {\Vert }_a\sim \Vert \cdot {\Vert }_b$.

2. Then, there exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

   $$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

3. Choose $d_1=c_2$ and $d_2=c_1$.

4. Then, for every $\mathbf{v}\in \mathbb{V}$, we have

   $$\Vert \mathbf{v}{\Vert }_b\le d_1\Vert \mathbf{v}{\Vert }_a\text{ and }\Vert \mathbf{v}{\Vert }_a\le d_2\Vert \mathbf{v}{\Vert }_b.$$

5. Thus, $\Vert \cdot {\Vert }_b\sim \Vert \cdot {\Vert }_a$.

[Transitivity]

1. Let $\Vert \cdot {\Vert }_a\sim \Vert \cdot {\Vert }_b$ and $\Vert \cdot {\Vert }_b\sim \Vert \cdot {\Vert }_c$.

2. Then, there exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

   $$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

3. And, there exist constants $d_1,d_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

   $$\Vert \mathbf{v}{\Vert }_b\le d_1\Vert \mathbf{v}{\Vert }_c\text{ and }\Vert \mathbf{v}{\Vert }_c\le d_2\Vert \mathbf{v}{\Vert }_b.$$

4. Let $e_1=c_1{d}_1>0$ and $e_2=c_2{d}_2>0$.

5. Then, for every $\mathbf{v}\in \mathbb{V}$:

   $$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\le c_1(d_1\Vert \mathbf{v}{\Vert }_c)=(c_1{d}_1)\Vert \mathbf{v}{\Vert }_c=e_1\Vert \mathbf{v}{\Vert }_c.$$

6. Similarly, for every $\mathbf{v}\in \mathbb{V}$:

   $$\Vert \mathbf{v}{\Vert }_c\le d_2\Vert \mathbf{v}{\Vert }_b\le d_2(c_2\Vert \mathbf{v}{\Vert }_a)=(d_2{c}_2)\Vert \mathbf{v}{\Vert }_a=e_2\Vert \mathbf{v}{\Vert }_a.$$

7. Thus, $\Vert \cdot {\Vert }_a\sim \Vert \cdot {\Vert }_c$.

Theorem 4.59 (Norm equivance $⟹$ metric strong equivalence)

If two norms are equivalent, then there associated metrics are strongly equivalent.

Proof. Let $\mathbb{V}$ be a given vector space. Let $\Vert \cdot {\Vert }_a:\mathbb{V}\to \mathbb{R}$, and $\Vert \cdot {\Vert }_b:\mathbb{V}\to \mathbb{R}$ be norms defined on $\mathbb{V}$ which are equivalent.

Then, there exist constants $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$, we have

$$\Vert \mathbf{v}{\Vert }_a\le c_1\Vert \mathbf{v}{\Vert }_b\text{ and }\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a.$$

The associated metrics are given by

$$d_a(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}{\Vert }_a\text{ and }{d}_b(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}{\Vert }_b.$$

Clearly, for every $\mathbf{x},\mathbf{y}\in \mathbb{V}$:

$$d_a(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}{\Vert }_a\le c_1\Vert \mathbf{x}-\mathbf{y}{\Vert }_b=c_1{d}_b(\mathbf{x},\mathbf{y}).$$

Similarly,

$$d_b(\mathbf{x},\mathbf{y})=\Vert \mathbf{x}-\mathbf{y}{\Vert }_b\le c_2\Vert \mathbf{x}-\mathbf{y}{\Vert }_a=c_2{d}_a(\mathbf{x},\mathbf{y}).$$

Thus, there exist $c_1,c_2>0$ such that

$$d_a(\mathbf{x},\mathbf{y})\le c_1{d}_b(\mathbf{x},\mathbf{y})\text{ and }{d}_b(\mathbf{x},\mathbf{y})\le c_2{d}_a(\mathbf{x},\mathbf{y})\mathrm{\forall }\mathbf{x},\mathbf{y}\in \mathbf{x},\mathbf{y}\in \mathbb{V}.$$

Thus, the two associated metrics are strongly equivalent.

4.4.15.1. Norms on Finite Dimensional Spaces

Theorem 4.60 (Equivalence of norms on a finite dimensional vector space)

Let $\mathbb{V}$ be a finite dimensional vector space over the scalar field $\mathbb{F}$ where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. Then, all norms on $\mathbb{V}$ are equivalent.

Reaching this conclusion requires significant amount of work on the norms on Euclidean spaces which are discussed in detail in The Euclidean Space. Readers are advised to read the material on norms on ${\mathbb{R}}^n$ and the fact that all norms on ${\mathbb{R}}^n$ are equivalent before proceeding further.

Proof. If $0=\dim \mathbb{V}$, then there is only one norm and there is nothing to prove. So assume that $\dim \mathbb{V}>0$. Then, $\mathbb{V}$ is isomorphic to ${\mathbb{R}}^n$ for some $n$.

1. Let $L:{\mathbb{R}}^n\to \mathbb{V}$ be an isomorphism.

2. Now, if $\Vert \cdot \Vert$ is a norm on $\mathbb{V}$, then the function $\Vert \cdot {\Vert }_L:{\mathbb{R}}^n\to \mathbb{R}$ defined by

   $$\Vert \mathbf{x}{\Vert }_L=\Vert L(\mathbf{x})\Vert$$

   is a norm on ${\mathbb{R}}^n$.

3. Let $\Vert \cdot {\Vert }_a$ and $\Vert \cdot {\Vert }_b$ be two different norms on $\mathbb{V}$.

4. Let $\Vert \cdot {\Vert }_{L,a}$ and $\Vert \cdot {\Vert }_{L,b}$ be the corresponding induced norms on ${\mathbb{R}}^n$.

5. By Theorem 4.117, all norms on ${\mathbb{R}}^n$ are equivalent.

6. Hence, $\Vert \cdot {\Vert }_{L,a}$ and $\Vert \cdot {\Vert }_{L,b}$ are equivalent.

7. Hence, there exist constants $c_1,c_2>0$ such that

   $$\Vert \mathbf{x}{\Vert }_{L,a}\le c_1\Vert \mathbf{x}{\Vert }_{L,b}\text{ and }\Vert \mathbf{x}{\Vert }_{L,b}\le c_2\Vert \mathbf{x}{\Vert }_{L,a}$$

   holds true for every $\mathbf{x}\in {\mathbb{R}}^n$.

8. Then, for every $\mathbf{v}\in \mathbb{V}$

   $$\begin{array}{rl}\Vert \mathbf{v}{\Vert }_a& =\Vert L(L^{-1}(\mathbf{v})){\Vert }_a\\ & =\Vert L^{-1}(\mathbf{v}){\Vert }_{L,a}\\ & \le c_1\Vert L^{-1}(\mathbf{v}){\Vert }_{L,b}\\ & =c_1\Vert L(L^{-1}(\mathbf{v})){\Vert }_b\\ & =c_1\Vert \mathbf{v}{\Vert }_b.\end{array}$$

9. Similarly, for every $\mathbf{v}\in \mathbb{V}$

   $$\Vert \mathbf{v}{\Vert }_b\le c_2\Vert \mathbf{v}{\Vert }_a$$

   holds true.

10. Thus, the two norms are equivalent.

Since, the two norms chosen were arbitrary, hence all norms on $\mathbb{V}$ are equivalent.

Definition 4.69 (Norm topology)

Let $\mathbb{V}$ be a finite dimensional space. Since all norms on $\mathbb{V}$ are equivalent, hence, they induce the same topology (family of open sets, closed sets, compact sets). The topology induced by a norm (the collection of open sets) on a finite dimensional space is called its norm topology.

4.4.15.2. Continuity

Continuity of a function doesn’t depend on the choice of a norm within the class of all equivalent norms. In other words, if a function $f$ is continuous for a given norm, then it is continuous for all norms equivalent to it.

Theorem 4.61 (Continuity with equivalent norms)

Let $\mathbb{V}$ be a given vector space. Let $\Vert \cdot {\Vert }_a:\mathbb{V}\to \mathbb{R}$, and $\Vert \cdot {\Vert }_b:\mathbb{V}\to \mathbb{R}$ be norms defined on $\mathbb{V}$ which are equivalent. Let $d_a$ and $d_b$ be corresponding metrics. Let $(X,d)$ be a metric space.

Let $f:X\to \mathbb{V}$ be a (total) function.

Then, $f:(X,d)\to (\mathbb{V},d_a)$ is continuous if and only if $f:(X,d)\to (\mathbb{V},d_b)$ is continuous.

Similarly, let $g:\mathbb{V}\to X$ be a (total) function.

Then, $g:(\mathbb{V},d_a)\to (X,d)$ is continuous if and only if $g:(\mathbb{V},d_b)\to (X,d)$ is continuous.

Proof. By Theorem 4.59, the metrics $d_a$ and $d_b$ are strongly equivalent. By Theorem 3.28, they are equivalent. Thus, they determine the same topology on $\mathbb{V}$. Thus, a subset $A\subseteq \mathbb{V}$ is open in $(\mathbb{V},d_a)$ if and only if it is open in $(\mathbb{V},d_b)$.

Now, assume $f:(X,d)\to (\mathbb{V},d_a)$ to be continuous.

1. Let $A$ be an open set of $(\mathbb{V},d_a)$.

2. Then, $f^{-1}(A)$ is an open set of $(X,d)$ since $f:(X,d)\to (\mathbb{V},d_a)$ is continuous (Theorem 3.42).

3. But $A$ is also an open set of $(\mathbb{V},d_b)$ since both metrics determine same open sets.

4. Thus, whenever $A$ is an open set of $(\mathbb{V},d_b)$, $f^{-1}(A)$ is an open set of $(X,d)$.

5. Thus, $f:(X,d)\to (\mathbb{V},d_b)$ is continuous due to Theorem 3.42.

A similar reasoning shows that if $f:(X,d)\to (\mathbb{V},d_b)$ is continuous then $f:(X,d)\to (\mathbb{V},d_a)$ must be continuous too.

Combining, $f:(X,d)\to (\mathbb{V},d_a)$ is continuous if and only if $f:(X,d)\to (\mathbb{V},d_b)$ is continuous.

Now, assume $g:(\mathbb{V},d_a)\to (X,d)$ to be continuous.

1. Let $A$ be an open set of $(X,d)$.

2. Then, $g^{-1}(A)$ is an open set of $(\mathbb{V},d_a)$ since $g:(\mathbb{V},d_a)\to (X,d)$ is continuous (Theorem 3.42).

3. But $g^{-1}(A)$ is also an open set of $(\mathbb{V},d_b)$ since both metrics determine same open sets.

4. Thus, whenever $A$ is an open set of $(X,d)$, $g^{-1}(A)$ is an open set of $(\mathbb{V},d_b)$.

5. Thus, $g:(\mathbb{V},d_b)\to (X,d)$ is continuous due to Theorem 3.42.

A similar reasoning shows that if $g:(\mathbb{V},d_b)\to (X,d)$ is continuous then $g:(\mathbb{V},d_a)\to (X,d)$ must be continuous too.

Combining, $g:(\mathbb{V},d_a)\to (X,d)$ is continuous if and only if $g:(\mathbb{V},d_b)\to (X,d)$ is continuous.

Remark 4.9 (Continuity in finite dimensional spaces)

In the special case where $\mathbb{V}$ is finite dimensional, the continuity of $f$ or $g$ is independent of the choice of norm chosen on $\mathbb{V}$ since all norms are equivalent.

4.4.16. Linear Transformations in Finite Dimensional Spaces

Theorem 4.62 (Linear transformations in finite dimensional spaces are bounded)

Let $(\mathbb{V},\Vert \cdot {\Vert }_v)$ and $(\mathbb{W},\Vert \cdot {\Vert }_w)$ be normed linear spaces. Let $T:\mathbb{V}\to \mathbb{W}$ be a linear transformation.

If $\mathbb{V}$ is finite dimensional, then $T$ is bounded.

Proof. If $\dim \mathbb{V}=0$, then $\mathbb{V}=\{\mathbf{0}\}$ and any linear transformation is bounded. Hence, let $\dim \mathbb{V}>0$.

Since $\mathbb{V}$ is finite dimensional, we can choose a basis $\mathcal{B}=\{{\mathbf{e}}_1,\dots ,{\mathbf{e}}_n\}$ for $\mathbb{V}$.

1. Let $c_i=\Vert T({\mathbf{e}}_i){\Vert }_w$ for $i=1,\dots ,n$ and $c=max\{c_1,\dots ,c_n\}$.

2. Let $\mathbf{x}\in \mathbb{V}$.

3. It can be uniquely written as

   $$\mathbf{x}=t_1{\mathbf{e}}_1+\cdots +t_n{\mathbf{e}}_n.$$

4. Then,

   $$\begin{array}{rl}\Vert T(\mathbf{x}){\Vert }_w& =\Vert T(t_1{\mathbf{e}}_1+\cdots +t_n{\mathbf{e}}_n)\Vert \\ & =\Vert t_{1}T({\mathbf{e}}_1)+\cdots +t_{n}T({\mathbf{e}}_n)\Vert \\ & \le |t_1|\Vert T({\mathbf{e}}_1){\Vert }_w+\cdots +|t_n|\Vert T({\mathbf{e}}_n){\Vert }_w\\ & \le c(|t_1|+\cdots +|t_n|).\end{array}$$

5. Note that the function $\Vert \cdot {\Vert }_{\ast }:\mathbb{V}\to \mathbb{R}$ given by

   $$\Vert \mathbf{x}{\Vert }_{\ast }=|t_1|+\cdots +|t_n|$$

   is a norm on $\mathbb{V}$.

6. By Theorem 4.60, all norms on $\mathbb{V}$ are equivalent since $\mathbb{V}$ is finite dimensional.

7. Thus, there exists $c_1>0$ such that

   $$\Vert \mathbf{x}{\Vert }_{\ast }\le c_1\Vert \mathbf{x}{\Vert }_v$$

   for every $\mathbf{x}\in \mathbb{V}$.

8. Thus, $\Vert T(\mathbf{x}){\Vert }_w\le c{c}_1\Vert \mathbf{x}{\Vert }_v$ for every $\mathbf{x}\in \mathbb{V}$.

9. Thus, for every nonzero $\mathbf{x}\in \mathbb{V}$,

   $$\frac{\Vert T(\mathbf{x}){\Vert }_w}{\Vert \mathbf{x}{\Vert }_v}\le c{c}_1.$$

10. Thus, the set $S$ in (4.1) is bounded.

11. Thus, $T$ is bounded.

Theorem 4.63 (Linear transformations in finite dimensional spaces are continuous)

Let $(\mathbb{V},\Vert \cdot {\Vert }_v)$ and $(\mathbb{W},\Vert \cdot {\Vert }_w)$ be normed linear spaces. Let $T:\mathbb{V}\to \mathbb{W}$ be a linear transformation.

If $\mathbb{V}$ is finite dimensional, then $T$ is continuous as well as uniformly continuous as well as Lipschitz continuous.

Proof. By Theorem 4.62, $T$ is bounded. The rest follows from the characterization of continuous linear transformations on vector spaces in Theorem 4.55.

4.4.17. Linear Subspaces

Theorem 4.64 (Linear subspaces are closed)

Every subspace of a finite dimensional normed linear space $\mathbb{V}$ is a closed set.

Proof. Let $\mathbb{V}$ be a finite dimensional subspace equipped with a norm $\Vert \cdot \Vert$.

The trivial subspace $\{\mathbf{0}\}$ is closed since it is a singleton. The space $\mathbb{V}$ is closed by definition.

Let $W$ be a proper nontrivial subspace of $\mathbb{V}$. Then, there exists a linear transformation $T$ such that

$$T(\mathbf{x})=\mathbf{0}\mathrm{\forall }\mathbf{x}\in W;$$

i.e., $W$ is the kernel of $T$.

Then,

$$W=T^{-1}(\{\mathbf{0}\}).$$

By Theorem 4.63, $T$ is continuous.

And the set $\{\mathbf{0}\}$ is a singleton, hence closed.

Then, by Theorem 3.42 (5), $W=T^{-1}(\{\mathbf{0}\})$ is a closed set.

4.4.18. Completeness and Banach Spaces

Definition 4.70 (Banach space)

A normed space $\mathbb{V}$ that is complete with respect to the metric induced by its norm is called a Banach space.

In other words, $\mathbb{V}$ is a Banach space if every Cauchy sequence of $\mathbb{V}$ converges in $\mathbb{V}$.

Example 4.19 (Examples of Banach spaces)

The spaces in examples below have been described in detail elsewhere. Follow the links.

1. The Euclidean space ${\mathbb{R}}^n$ is complete with respect to the Euclidean norm. Thus, it is a Banach space.

2. The space of bounded real valued functions $B(X)$ over a nonempty set $X$ equipped with sup norm is complete.

Theorem 4.65 ($n$-dim normed linear spaces are complete)

Let $\mathbb{V}$ be an $n$-dimensional vector space over $\mathbb{F}$ where $\mathbb{F}$ is either $\mathbb{R}$ or $\mathbb{C}$. Let $\mathbb{V}$ be equipped with a norm $\Vert \cdot \Vert :\mathbb{V}\to \mathbb{R}$ making it a normed linear space. Then $\mathbb{V}$ is complete.

In other words, every finite dimensional normed linear space is complete.

Proof. $\mathbb{V}$ is a normed linear space with $n=\dim \mathbb{V}$.

1. Let $\mathcal{B}=\{{\mathbf{e}}_1,\dots ,{\mathbf{e}}_n\}$ be a basis for $\mathbb{V}$.

2. For any $\mathbf{v}\in \mathbb{V}$, we can represent it in the basis $\mathcal{B}$ as

   $$\mathbf{v}=v_1{\mathbf{e}}_1+\cdots +v_n{\mathbf{e}}_n.$$

3. We can define the $\Vert \cdot {\Vert }_1$ norm as

   $$\Vert \mathbf{v}{\Vert }_1=|v_1|+\cdots +|v_n|.$$

4. By Theorem 4.60, all norms on $\mathbb{V}$ are equivalent.

5. Thus, $\Vert \cdot {\Vert }_1$ and $\Vert \cdot \Vert$ are equivalent.

6. Thus, there are $c_1,c_2>0$ such that for every $\mathbf{v}\in \mathbb{V}$,

   $$c_1\Vert \mathbf{v}{\Vert }_1\le \Vert \mathbf{v}\Vert \le c_2\Vert \mathbf{v}{\Vert }_1.$$

7. Let $\{{\mathbf{v}}_k\}$ be a Cauchy sequence of $\mathbb{V}$.

8. Let $ϵ>0$.

9. Since $\{{\mathbf{v}}_k\}$ is a Cauchy sequence of $\mathbb{V}$, hence there exists $N$ such that for every $k,l>N$,

   $$\Vert {\mathbf{v}}_k-{\mathbf{v}}_l\Vert <ϵ.$$

10. Then,

    $$\begin{array}{rl}ϵ& >\Vert {\mathbf{v}}_k-{\mathbf{v}}_l\Vert \\ & \ge c_1\Vert {\mathbf{v}}_k-{\mathbf{v}}_l{\Vert }_1\\ & =c_1\sum _{i=1}^n|v_{ki}-v_{li}|\\ & \ge c_1|v_{ki}-v_{li}|\end{array}$$

    for every $i\in 1,\dots ,n$.

11. Hence $\{v_{ki}\}$ is a Cauchy sequence of $\mathbb{F}$ for every $i\in 1,\dots ,n$.

12. Since both $\mathbb{R}$ and $\mathbb{C}$ are complete, hence $\{v_{ki}\}$ is a convergent sequence of $\mathbb{F}$ for every $i\in 1,\dots ,n$.

13. Thus, there exists $u_i=\underset{n\to \mathrm{\infty }}{lim}{v}_{ki}$ for every $i\in 1,\dots ,n$.

14. Now, let $\mathbf{u}=(u_1,\dots ,u_n)$.

15. Then,

    $$\begin{array}{rl}\Vert {\mathbf{v}}_k-\mathbf{u}\Vert & \le c_2\Vert {\mathbf{v}}_k-\mathbf{u}{\Vert }_1\\ & =c_2\sum _{i=1}^n|v_{ki}-u_i|.\end{array}$$

16. Thus,

    $$\begin{array}{rl}\underset{k\to \mathrm{\infty }}{lim}\Vert {\mathbf{v}}_k-\mathbf{u}\Vert & =\underset{k\to \mathrm{\infty }}{lim}{c}_2\sum _{i=1}^n|v_{ki}-u_i|\\ & =c_2\sum _{i=1}^n\underset{k\to \mathrm{\infty }}{lim}|v_{ki}-u_i|=0.\end{array}$$

17. Thus, $\underset{k\to \mathrm{\infty }}{lim}{\mathbf{v}}_k=\mathbf{u}$.

18. Thus, $\{{\mathbf{v}}_k\}$ is convergent.

19. Thus, every Cauchy sequence of $\mathbb{V}$ is convergent.

20. Thus, $\mathbb{V}$ is complete.

4.4.19. Compact Sets

In this subsection, we pay special attention to the compact subsets of $n$-dimensional real normed linear spaces. We show that in such spaces, closed and bounded sets are compact. We provides some results related to set arithmetic for compact sets. We further establish the Bolzano Weierstrass theorems for such spaces. Recall from Definition 3.57 that a set has Bolzano-Weierstrass property if every sequence of the set has a convergent subsequence that converges to a point in the set.

Theorem 4.66 (Compact = Closed and Bounded in $n$-dim)

Let $\mathbb{V}$ be a real $n$-dimensional normed linear space. Let $S$ be a subset of $\mathbb{V}$.

Then, $S$ is compact if and only if $S$ is closed and bounded.

Proof. By Theorem 3.76, every compact set is closed and bounded. Thus, if $S$ is compact then $S$ is closed and bounded.

For the converse, assume that $S$ is closed and bounded.

1. By Theorem 4.65, both ${\mathbb{R}}^n$ and $\mathbb{V}$ are complete.

2. Let $T:{\mathbb{R}}^n\to \mathbb{V}$ be an isomorphism.

3. Then, for any $\mathbf{v}\in {\mathbb{R}}^n$, a function $\Vert \cdot {\Vert }_T:{\mathbb{R}}^n\to \mathbb{R}$ given by

   $$\Vert \mathbf{v}{\Vert }_T=\Vert T(\mathbf{v})\Vert$$

   defines a norm on ${\mathbb{R}}^n$.

4. The norm $\Vert \cdot {\Vert }_T$ is equivalent to the Euclidean norm on ${\mathbb{R}}^n$ due to Theorem 4.60.

5. Also, $T:({\mathbb{R}}^n,\Vert \cdot {\Vert }_T)\to (\mathbb{V},\Vert \cdot \Vert )$ is an isometry

6. Accordingly $T$ is uniformly continuous by Theorem 3.54.

7. Since $T$ is an isomorphism, it is bijective.

8. Thus, $T$ is a homeomorphism by Theorem 3.55.

9. Then, $T^{-1}$ is also an isometry.

10. Then, $T^{-1}(S)$ is bounded in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_T)$ since $S$ is bounded in $(\mathbb{V},\Vert \cdot \Vert )$ and $T^{-1}$ is an isometry.

11. By Theorem 4.57, $T^{-1}(S)$ is also bounded in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_2)$.

12. Also, $T^{-1}(S)$ is closed in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_T)$. since $S$ is closed in $(\mathbb{V},\Vert \cdot \Vert )$ and $T$ is continuous (see Theorem 3.42).

13. Since equivalent norms (metrics) determine same topology, hence $T^{-1}(S)$ is closed in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_2)$ also.

14. Thus, $T^{-1}(S)$ is closed and bounded in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_2)$.

15. By Heine-Borel theorem, $T^{-1}(S)$ is a compact set in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_2)$.

16. By Theorem 3.90, $T^{-1}(S)$ is a compact set in $({\mathbb{R}}^n,\Vert \cdot {\Vert }_T)$ also.

17. Then, $S=T(T^{-1}(S))$ is also compact in $(\mathbb{V},\Vert \cdot \Vert )$ since a homeomorphism preserves compactness (see Theorem 3.80).

Theorem 4.67 (Sum of compact sets is compact)

Let $\mathbb{V}$ be a real $n$-dimensional normed linear space. Let $A,B\subseteq \mathbb{V}$ be compact subsets of $\mathbb{V}$. Then, their sum $A+B$ is compact.

Proof. We proceed as follows

1. Both $A$ and $B$ are compact. Hence, they are closed and bounded.

2. By Theorem 4.50, $A+B$ is closed.

3. Let $\Vert \mathbf{a}\Vert \le M_a$ for every $\mathbf{a}\in A$.

4. Let $\Vert \mathbf{b}\Vert \le M_b$ for every $\mathbf{b}\in B$.

5. Let $\mathbf{x}\in A+B$.

6. Then, there exists $\mathbf{a}\in A$ and $\mathbf{b}\in B$ such that $\mathbf{x}=\mathbf{a}+\mathbf{b}$.

7. Then,

   $$\Vert \mathbf{x}\Vert =\Vert \mathbf{a}+\mathbf{b}\Vert \le \Vert \mathbf{a}\Vert +\Vert \mathbf{b}\Vert \le M_a+M_b.$$

8. Thus, $\Vert \mathbf{x}\Vert \le M_a+M_b$ for every $\mathbf{x}\in A+B$.

9. Thus, $A+B$ is bounded.

10. Since $A+B$ is closed and bounded, hence $A+B$ is compact due to Theorem 4.66.

Theorem 4.68 (Bolzano Weierstrass theorem for bounded subsets)

Let $\mathbb{V}$ be a real $n$-dimensional normed linear space. Let $A$ be a bounded subset of $\mathbb{V}$. Then, every sequence of $A$ has a convergent subsequence.

If $A$ is closed, then the subsequence converges in $A$ itself. Otherwise, the subsequence converges in $\mathrm{cl}A$.

Proof. We are given that $A$ is bounded.

1. Then, there exist $M>0$ such that:

   $$\Vert \mathbf{x}\Vert \le M\mathrm{\forall }\mathbf{x}\in A.$$

2. Thus, $A\subseteq B[\mathbf{0},M]$.

3. $B[\mathbf{0},M]$ is a closed and bounded subset of $\mathbb{V}$.

4. $\mathrm{cl}A\subseteq B[\mathbf{0},M]$ since $\mathrm{cl}A$ is the smallest closed set containing $A$.

5. Thus, $\mathrm{cl}A$ is closed and bounded.

6. By Theorem 4.66, $\mathrm{cl}A$ is compact.

7. Let $\{{\mathbf{x}}_k\}$ be a sequence of $A$. Then, it is also a sequence of $\mathrm{cl}A$.

8. By Theorem 3.75, $\{{\mathbf{x}}_k\}$ has a subsequence that converges in $\mathrm{cl}A$.

9. If $A$ is closed, then $\mathrm{cl}A=A$ and we are done.

Theorem 4.69 (Bolzano Weierstrass theorem for bounded sequences)

Let $\mathbb{V}$ be a real $n$-dimensional normed linear space. Every bounded sequence of $\mathbb{V}$ has a convergent subsequence.

Proof. Let $\{{\mathbf{x}}_k\}$ be a bounded sequence of $\mathbb{V}$.

1. Then there exists a closed ball $B[\mathbf{0},M]$ such that $\{{\mathbf{x}}_k\}\subset B[\mathbf{0},M]$.

2. $B[\mathbf{0},M]$ is closed and bounded.

3. By Theorem 4.66, $B[\mathbf{0},M]$ is compact.

4. By Theorem 3.75, $\{{\mathbf{x}}_k\}$ has a subsequence that converges in $B[\mathbf{0},M]$.