(sec:la:normed-spaces)=
# Normed Linear Spaces
A norm is a real number attached to every vector.
Norm is a generalization of the notion of length.
Adding a norm to a vector space makes it a normed
linear space with rich topological properties
as a norm induces a
{prf:ref}`distance function ` (a metric)
on the vector space converting it into a
{prf:ref}`metric space `
with an algebraic structure.
We can identify open sets,
closed sets, bounded sets, compact sets, etc..
We can introduce the notion of continuity
on the functions defined from one normed linear space
to another metric space (which could be a normed
linear space or just the real line).
The algebraic structure allows us to consider
operations like translations, scaling, rotation,
and general affine transformations on sets of vectors
which can be thought of as geometrical objects.
There are many ways to define norms on a vector space.
A question arises as to which norms are equivalent
to each other in the sense that they determine
the same topology on the underlying vector space.
It turns out that in finite dimensional vector
spaces, all norms are equivalent.
Linear transformations play a central role
in linear algebra. Linear transformations
are generally unbounded by the usual notion
of boundedness of a function (where we say that
the range of the function is bounded).
However, there is another way to examine the
boundedness of linear transformations by
examine the ratio of norms of the input to the
transformation and the output generated by it.
For bounded linear transformations, the ratio
is bounded. For matrices, this is the largest
singular value.
Not all linear transformations are continuous,
but linear transformations on finite dimensional
spaces are continuous. In fact, for linear
transformations, boundedness, continuity,
uniform continuity and Lipschitz continuity
turn out to be the same thing.
A normed linear space that is
{prf:ref}`complete ` is called
a Banach space.
We restrict our attention to real vector spaces and complex vector spaces.
Thus, the field $\FF$ can be either $\RR$ or $\CC$.
## Norm
```{index} Norm
```
````{prf:definition} Norm
:label: def-la-norm
A *norm* over a $\FF$-vector space $\VV$ is any real valued function
$\| \cdot \| : \VV \to \RR$ mapping $ \bv \mapsto \| \bv\|$
satisfying following properties:
1. [Positive definiteness]
$$
\| \bv\| \geq 0 \quad \forall \bv \in \VV \text{ and } \| \bv\| = 0 \iff \bv = \bzero.
$$
1. [Positive homogeneity]
$$
\| \alpha \bv \| = | \alpha | \| \bv \| \quad \forall \alpha \in \FF; \forall \bv \in \VV.
$$
1. [Triangle inequality]
$$
\| \bv_1 + \bv_2 \| \leq \| \bv_1 \| + \| \bv_2 \| \quad \forall \bv_1, \bv_2 \in \VV.
$$
````
The norm doesn't change on negation.
```{prf:remark} Negation and norm
:label: res-la-ns-negation-norm
$$
\| - \bv \| = \| (-1) \bv \| = | - 1 | \| \bv \| = \| \bv \| \Forall \bv \in \VV.
$$
```
There is a different form of triangle inequality which is quite
useful in the following analysis. It can be derived from the
properties of a norm.
```{prf:theorem} Triangle inequality II
:label: res-la-ns-triangle-inequality-2
$$
| \| \bx \| - \| \by \| | \leq \| \bx - \by \| \Forall \bx, \by \in \VV.
$$
```
```{prf:proof}
Putting $\bv_1 = \bx$ and $\bv_2 = \by - \bx$ in the triangle inequality, we get:
$$
\begin{aligned}
& \| \bx + \by - \bx \| \leq \| \bx \| + \| \by - \bx \| \\
& \iff \| \by \| \leq \| \bx \| + \| \by - \bx \| \\
& \iff \| \by \| - \| \bx \| \leq \| \by - \bx \| = \| \bx - \by \|.
\end{aligned}
$$
Interchanging $\bx$ and $\by$ in previous inequality, we get:
$$
\| \bx \| - \| \by \| \leq \| \bx - \by \|.
$$
Combining the two inequalities, we get:
$$
| \| \bx \| - \| \by \| | \leq \| \bx - \by \| \Forall \bx, \by \in \VV.
$$
```
```{prf:theorem} Triangle inequality for vector differences
:label: def-la-ns-distance-triangle-inequality
The triangle inequality is equivalent to the following property:
$$
\| \bx - \by \| \leq \| \bx - \bz \| + \| \bz - \by \| \Forall \bx, \by, \bz \in \VV.
$$
```
```{prf:proof}
Start with
$$
\| \bv_1 + \bv_2 \| \leq \| \bv_1 \| + \| \bv_2 \|.
$$
Put $\bv_1 = \bx - \bz $ and $\bv_2 = \bz - \by$. We get:
$$
\| \bx - \bz + \bz - \by \| \leq \| \bx - \bz \| + \| \bz - \by \|
\iff
\| \bx - \by \| \leq \| \bx - \bz \| + \| \bz - \by \|.
$$
For the converse, start with:
$$
\| \bx - \by \| \leq \| \bx - \bz \| + \| \bz - \by \|
$$
Put $\bv_1 = \bx - \bz$ and $\bv_2 = \bz - \by$. Then, $\bv_1 + \bv_2 = \bx - \by$.
We get:
$$
\| \bv_1 + \bv_2 \| \leq \| \bv_1 \| + \| \bv_2 \|.
$$
```
## Normed Linear Space
```{index} Normed linear space, Normed space
```
````{prf:definition} Normed linear space
:label: def-la-normed-linear-space
An $\FF$-vector space $\VV$ equipped with a norm $\| \| : \VV \to \RR$ is known
as a *normed linear space*. Other common terms are
*normed vector space* or simply *normed space*.
````
```{prf:example} Norm for the trivial vector space
:label: ex-la-ns-trivial
The trivial vector space contains a single vector
which is the zero vector; i.e., $\VV = \{ \bzero \}$.
A function $f : \VV \to \RR$ given by $f(\bzero) = 0$
satisfies all the properties of a norm.
It is easy to see that this is the only possible norm
for the trivial vector space.
```
```{prf:remark}
:label: rem-la-ns-ignore-trivial
We will assume that the vector space is non-trivial;
i.e., different from $\{ \bzero \}$ in this section.
Wherever necessary, we will provide details about
how a particular result is valid for the trivial vector space also.
```
```{prf:example} Real line $\RR$
:label: ex-la-ns-real-line
Recall from {prf:ref}`ex-field-is-vector-space` that
$\RR$ is a vector space by itself.
The function $ f : \RR \to \RR$ given by
$$
f(x) = | x |
$$
satisfies all the requirements of a norm.
Thus, $(\RR, | \cdot |)$ is a normed linear space.
```
```{prf:example} Euclidean space $\RR^n$
:label: ex-la-ns-euclidean
The Euclidean space $\RR^n$ can be equipped with
a variety of norms. They are discussed in detail
in {ref}`sec:la:real-euclidean-space`.
1. {prf:ref}`The Euclidean norm `
1. $\ell_p$ {prf:ref}`norms `
When we propose a norm for a vector space, we justify
it by showing that it satisfies all the properties
of a norm.
{prf:ref}`res-la-euclid-lp-norm-just`
justifies that all $\ell_p$ norms on $\RR^n$
are indeed norms.
```
## Normed Space as a Metric Space
```{index} Metric; induced by norm
```
```{prf:definition} Metric induced by a norm
:label: def-la-norm-induced-metric
Every norm $\| \cdot \| : \VV \to \RR$ induces a
{prf:ref}`metric ` defined as:
$$
d(\bx , \by) = \| \bx - \by \| \Forall \bx, \by \in \VV.
$$
```
```{prf:theorem} Norm induced metric justification
:label: res-la-ns-metric-justification
The metric $d : \VV \times \VV \to \RR$
induced by a norm $\| \cdot \| : \VV \to \RR$ is indeed
a metric satisfying all the properties of a
{prf:ref}`metric `( distance function).
```
```{prf:proof}
We proceed as follows:
1. Non-negativity: $d(\bx, \by) \geq 0$ since $\| \cdot \|$ is positive definite.
1. Identity of indiscernibles.
1. Assume $d(\bx, \by) = 0$.
1. Then, $\| \bx - \by \| = 0$.
1. Thus, $\bx = \by$ since $\| \cdot \|$ is positive definite.
1. Now, assume $\bx = \by$.
1. Then, $d(\bx, \by) = \| \bx - \by \| = \| \bzero \| = 0$ since $\| \cdot \|$ is positive definite.
1. Symmetry: $d(\bx, \by) = \| \bx - \by \| = \| (-1)(\by - \bx) \| = |-1| \| \by - \bx \| = d(\by, \bx)$ using the positive homogeneity property of $\| \cdot \|$.
1. Triangle inequality: See {prf:ref}`def-la-ns-distance-triangle-inequality` above.
```
```{prf:remark}
:label: rem-la-ns-notation-norm-dist
We will use the notation $\| \cdot \|$ to denote both the
norm and the metric induced by the norm.
```
```{index} Metric space; from a norm
```
```{prf:definition} Metric space from a norm
The normed space $\VV$ equipped with the metric
induced by the norm as defined in {prf:ref}`def-la-norm-induced-metric`
becomes a *metric space* $(\VV, \| \cdot \|)$.
If the norm and the induced metric are clear from the context,
then we shall simply write it as $\VV$.
```
```{prf:theorem} Translation invariance
:label: res-la-ns-metric-translation-invariant
The metric induced by a norm is translation invariant.
For any $\bu, \bv, \bw \in \VV$:
$$
d(\bu, \bv) = d(\bu + \bw, \bv + \bw).
$$
```
```{prf:proof}
Expanding from definition:
$$
d(\bu + \bw, \bv + \bw) = \| \bu + \bw - (\bv + \bw) \|
= \| \bu - \bv \| = d(\bu, \bv).
$$
```
The topology of a general metric space is discussed in detail in
{ref}`sec:ms:metric-topology` and sections thereafter.
In this section, we discuss results which are specific
to normed linear spaces as they take advantage of
the additional structure provided by the vector space.
1. There is a special zero vector $\bzero \in \VV$.
It provides a reference point to define unit balls.
1. Vectors in $\VV$ can be added, subtracted and scaled.
Thus, general balls can be described in terms of unit
balls.
1. It is possible to introduce the notion of translation
of sets. Recall that the metric induced by the norm
is {prf:ref}`translation invariant `.
1. Sets of vectors in a vector space can be added/subtracted/scaled
since the underlying vectors can be.
This produces a number of interesting phenomena.
## Balls
An {prf:ref}`open ball ` in
a normed space is defined analogously as:
$$
B(\ba,r) = \{ \bx \in \VV \ST \| \bx - \ba \| < r \}.
$$
A {prf:ref}`closed ball ` in
a normed space is defined analogously as:
$$
B[\ba,r] = \{ \bx \in \VV \ST \| \bx - \ba \| \leq r \}.
$$
We sometimes use the notation
$B_{\| \cdot \|}(\ba,r)$
and $B_{\| \cdot \|}[\ba,r]$
to identify the specific norm being used to
describe the open and closed balls.
```{index} Normed linear space; unit ball
```
```{prf:definition} Unit ball
:label: def-la-unit-ball
A ball centered at origin $\bzero \in \VV$ is
called a *unit ball* if its radius is $1$.
$B[\bzero, 1]$ denotes a *closed unit ball*
and $B(\bzero, 1)$ denotes an *open unit ball*.
The open unit ball is often written simply as $B$.
```
```{prf:observation} Ball arithmetic
:label: res-la-ns-ball-arithmetic
Following the notation in {prf:ref}`def-vs-set-arithmetic`,
a closed ball can be expressed in terms of closed unit ball as:
$$
B[\bx, r] = \bx + r B[\bzero, 1].
$$
Similarly, any open ball can be expressed in terms of
the open unit ball as:
$$
B(\bx, r) = \bx + r B(\bzero, 1).
$$
```
```{prf:theorem} Ball addition and subtraction
:label: res-la-unit-ball-sum
Let $\VV$ be a normed linear space and $B$ be the unit open ball.
Then,
$$
B + B = B - B = 2B.
$$
```
```{prf:proof}
Due to {prf:ref}`res-vs-set-arithmetic-props`,
$2 B \subseteq B + B$.
We now show that $B + B \subseteq 2 B$.
1. Let $\ba \in B + B$.
1. Then, $\ba = \bb + \bc$ such that $\bb \in B$ and $\bc \in B$.
1. Now,
$
\| \ba \| = \| \bb + \bc \| \leq \| \bb \| + \| \bc \| < 1 + 1 = 2.
$
1. Thus, $\ba \in 2 B$.
Thus, $B + B = 2B$.
Next, note that $B$ is {prf:ref}`symmetric `;
i.e. $B = -B$.
Thus,
$$
B - B = B + (-B) = B + B = 2 B.
$$
```
```{prf:theorem} Moving a point into a ball
:label: res-la-ns-move-vec-to-ball
Let $\VV$ be a normed linear space. Let $B(\bx, r)$
be an open ball in $\VV$.
Then, every point in $\VV$ can be translated and scaled
into a point which is inside the ball $B(\bx, r)$.
```
```{prf:proof}
Let $\bv \in \VV$.
1. If $\bv = \bzero$, then we can simply translate it
by $\bx$ to put it inside $B(\bx, r)$.
1. Now, assume that $\bv \neq \bzero$. Then, $\| \bv \| \neq 0$.
1. Let $\by = \bx + \frac{r}{2 \| \bv \|} \bv$.
1. Then,
$$
\| \by - \bx \| = \left \| \frac{r}{2 \| \bv \|} \bv \right \|
= \frac{r}{2} < r.
$$
1. Thus, $\by \in B(\bx, r)$.
```
In the following, $B$ means the open unit ball $B(\bzero, 1)$.
## Interior
```{prf:theorem} Interior points in a normed space
:label: res-la-interior-point-ball
Let $A$ be a subset of a normed linear space $\VV$.
Then, $\bx \in \interior A$ if and only if there
exists $r > 0$ such that
$$
\bx + r B \subseteq A.
$$
```
```{prf:proof}
From {prf:ref}`def-ms-interior-point`,
$\bx$ is an interior point of $A$ if
there exists an $r > 0$ such that the open ball
$B(\bx, r) \subseteq A$.
But, as per algebraic notation $B(\bx, r) = \bx + r B$.
```
```{prf:theorem} Interior in a normed space
:label: res-la-interior
Let $A$ be a subset of a normed linear space $\VV$.
The interior of $A$ is given by
$$
\interior A = \{ \bx \ST \exists r > 0, \bx + r B \subseteq A \}.
$$
```
```{prf:proof}
This follows from the fact that the interior is a collection
of all the interior points of $A$.
```
```{prf:theorem} Subspaces and interior
:label: res-la-ns-nonempty-interior
Let $\VV$ be a normed vector space. The only
subspace of $\VV$ that has a nonempty interior
is $\VV$ itself.
```
```{prf:proof}
Let $S$ be a subspace of $\VV$.
1. Assume that $S$ has a nonempty interior.
1. Then, there is an interior point $\bx \in S$.
1. Thus, there is an open ball $B(\bx, r) \subseteq S$.
1. By {prf:ref}`res-la-ns-move-vec-to-ball`,
any nonzero point $\bv \in \VV$ can be moved into
the ball $B(\bx, r)$ as
$$
\by = \bx + \frac{r}{2 \| \bv \|} \bv.
$$
1. We have $\by \in S$.
1. But a subspace is closed under vector addition and scalar multiplication.
1. Thus,
$$
\bv = \frac{2 \| \bv \|}{r} (\by - \bx) \in S.
$$
1. Thus, $\bv \in \VV$ implies $\bv \in S$.
1. Thus, $\VV = S$.
Thus, the only subspace which has a nonempty interior is $\VV$
itself.
```
```{prf:corollary} Interior of proper subspaces
:label: res-la-proper-subspace-empty-interior
Every proper subspace of a normed linear space $\VV$
has an empty interior.
```
## Closure
```{prf:theorem} Closure points in a normed space
:label: res-la-closure-point-ball
Let $A$ be a subset of a normed linear space $\VV$.
Then, $\bx \in \closure A$ if and only if
$$
\bx \in A + r B \Forall r > 0.
$$
```
```{prf:proof}
Recall that
$$
A + r B = \bigcup_{\ba \in A} \ba + rB.
$$
Thus, $\bx \in A + r B$ if and only if
there exists $\ba \in A$ such that $\bx \in \ba + r B$.
Assume that $\bx \in A + r B \Forall r > 0$.
1. Thus, for every $r > 0$, there exists $\ba \in A$ (depending on $r$)
such that $\bx \in \ba + r B = B(\ba, r)$.
1. Thus, for every $r > 0$, there exists $\ba \in A$ (depending on $r$)
such that $d(\bx, \ba) < r$.
1. Thus, for every $r > 0$, there exists $\ba \in A$ (depending on $r$)
such that $\ba \in B(\bx, r)$.
1. Thus, $A \cap B(\bx, r) \neq \EmptySet$ for every $r > 0$.
1. Thus, $\bx$ is a closure point of $A$.
For the converse, assume that $\bx$ is a closure point of $A$.
1. Then, $A \cap B(\bx, r) \neq \EmptySet$ for every $r > 0$.
1. Thus, for every $r > 0$, there exists $\ba \in A$ (depending on $r$)
such that $\ba \in B(\bx, r)$.
1. Thus, for every $r > 0$, there exists $\ba \in A$ (depending on $r$)
such that $d(\bx, \ba) < r$.
1. Thus, for every $r > 0$, there exists $\ba \in A$ (depending on $r$)
such that $\bx \in B(\ba, r) = \ba + r B$.
1. Thus, for every $r > 0$,
$\bx \in A + r B$.
```
```{prf:theorem} Closure in a normed space
:label: res-la-closure
Let $A$ be a subset of a normed linear space $\VV$.
Then, the closure of $A$ is given by
$$
\closure A = \bigcap_{r > 0} (A + r B).
$$
```
```{prf:proof}
From previous result, a point $\bx$ is a closure point of $A$
if and only if
$$
\bx \in \bigcap_{r > 0} (A + r B).
$$
The result follows from the fact that the closure of $A$
is the collection of all its closure points.
```
## Continuity
```{index} Normed linear space; continuity
```
```{prf:definition} Continuity in normed linear spaces
:label: def-la-na-continuous-function
Let $(\VV, \| \cdot \|_v)$ and $(\WW, \| \cdot \|_w)$ be
normed linear spaces.
A function $f: \VV \to \WW$ between the two normed spaces
is said to be *continuous at a point* $\ba \in \dom f$
if for every $\epsilon > 0$, there exists $\delta > 0$
(depending on $\epsilon$ and $\ba$) such that
for all $\bx \in \dom f$
$$
\|\bx - \ba\|_v < \delta \implies \| f(\bx) - f(\ba)\|_w < \epsilon
$$
holds true.
$f$ is said to be *continuous* on $A \subseteq \dom f$
if $f$ is continuous at every point of $A$.
The function $f$ is said to be *uniformly continuous*
on $A \subseteq \dom f$
if for every $\epsilon > 0$, there exists some $\delta > 0$
(depending on $\epsilon$) such that
$$
\| f(\bx) - f(\by) \|_w < \epsilon
\text{ whenever }
\| \bx - \by \|_v < \delta \text{ and } \bx, \by \in A.
$$
```
These definitions are just adapted from the
corresponding definitions for metric spaces
for convenience.
### Norms
```{prf:theorem} Norms are uniformly continuous
:label: res-la-ns-norm-is-continuous
A function $\| \cdot \| : \VV \to \RR$ satisfying
all the properties of a {prf:ref}`norm `
is {prf:ref}`uniformly continuous `
in the metric space induced by the norm.
```
```{prf:proof}
Let $\bx, \by \in \VV$.
Assume $d(\bx, \by) < \delta$.
Choose $\epsilon = \delta$.
Now, due to {prf:ref}`res-la-ns-triangle-inequality-2`:
$$
d(\bx, \by) = \| \bx - \by \| \geq | \| \bx \| - \| \by \| |.
$$
Thus,
$$
d(\bx, \by) < \delta \implies | \| \bx \| - \| \by \| | < \delta = \epsilon.
$$
Thus, $\| \cdot \|$ is uniformly continuous.
```
### Translations
```{prf:theorem} Translations are uniformly continuous
:label: res-la-ns-translation-is-continuous
Let $(\VV, \| \cdot \|)$ be a normed linear space.
Let $\ba \in \VV$ be some fixed vector.
Let a translation map $g_a : \VV \to \VV$ be defined as
$$
g_a = \bx + \ba \Forall \bx \in \VV.
$$
Then, $g_a$ is uniformly continuous.
```
```{prf:proof}
Let $\epsilon > 0$ and $\delta = \epsilon$.
Let $\bx, \by \in \VV$.
Note that
$$
\| g_a (\bx) - g_a(\by) \|
= \| \bx + \ba - (\by + \ba) \|
= \| \bx - \by \|.
$$
Thus, $\| \bx - \by \| < \delta$ implies
$\| g_a (\bx) - g_a(\by) \| < \delta = \epsilon$.
Thus, $g_a$ is uniformly continuous.
```
```{prf:theorem} Translations preserve topology
:label: res-la-ns-translation-preserve-topology
Translations preserve the topology.
Let $(\VV, \| \cdot \|)$ be a normed linear space.
Let $\ba \in \VV$ be some fixed vector.
Let a translation map $g_a : \VV \to \VV$ be defined as
$$
g_a = \bx + \ba \Forall \bx \in \VV.
$$
Then,
1. $A$ is open if and only if $g_a(A)$ is open.
1. $A$ is closed if and only if $g_a(A)$ is closed.
1. $A$ is compact if and only if $g_a(A)$ is compact.
1. $A$ has an empty interior if and only if $g_a(A)$ has an empty interior.
```
```{prf:proof}
We note that the inverse of $g_a$ is given by
$g_{-a}$ which is also a translation.
Consequently, both $g_a$ and $g_{-a}$ are
uniformly continuous.
Preservation of open and closed sets are simple applications of
{prf:ref}`res-ms-continuous-function-characterization`
(characterization of continuous functions).
Preservation of compactness is
an application of {prf:ref}`res-ms-compact-continuous-map`
(continuous images of compact sets are compact).
Assume that $A$ has an interior point $\bx \in A$.
1. Then, there exists a ball $B(\bx, r) \subseteq A$.
1. Then, the ball $B(\bx, r) + \ba = B(\bx + \ba, r) \subseteq A + \ba$.
1. Thus, $\bx + ba$ is an interior point of $A + \ba = g_a(A)$.
Thus, if $g_a(A)$ has an empty interior then $A$ must
have an empty interior. The converse can be proved
by using the inverse translation $g_{-a}$.
```
### Scalar Multiplication
```{prf:theorem} Scalar multiplication is uniformly continuous
:label: res-la-ns-scaling-is-continuous
Let $(\VV, \| \cdot \|)$ be a normed linear space.
Let $t \in \FF$.
Let a scalar multiplication map $g_t : \VV \to \VV$ be defined as
$$
g_t = t \bx \Forall \bx \in \VV.
$$
Then, $g_t$ is uniformly continuous.
```
```{prf:proof}
If $t=0$, then $g_t(\bx) = \bzero$ for all $\bx \in \VV$
so it is trivially uniformly continuous.
Assume $t \neq 0$.
1. Let $\epsilon > 0$.
1. Choose $ \delta = \frac{\epsilon}{|t|}$.
1. Now, for any $\bx, \by \in \VV$
$$
\| g_t(\bx) - g_t(\by)\|
= \| t \bx - t \by \|
= \| t (\bx - \by)\|
= |t | \| \bx - \by \|.
$$
1. Thus, $\| \bx - \by \| < \delta$ implies
$$
\| g_t(\bx) - g_t(\by)\| < | t | \delta = \epsilon.
$$
1. Thus, $g_t$ is uniformly continuous.
```
```{prf:theorem} Scalar multiplication preserves topology
:label: res-la-ns-scaling-homeomorphism
Let $(\VV, \| \cdot \|)$ be a normed linear space.
Let $t \in \FF$ be nonzero (i.e., $t \neq 0$).
Let a scalar multiplication map $g_t : \VV \to \VV$ be defined as
$$
g_t = t \bx \Forall \bx \in \VV.
$$
Then, $g_t$ is a
{prf:ref}`homeomorphism `.
Consequently,
1. $A$ is open if and only if $g_t(A)$ is open.
1. $A$ is closed if and only if $g_t(A)$ is closed.
1. $A$ is compact if and only if $g_t(A)$ is compact.
1. $A$ has an empty interior if and only if $g_t(A)$ has an empty interior.
```
```{prf:proof}
We first show that $g_t$ is a homeomorphism.
Since $t \neq 0$, let $r = \frac{1}{t}$.
1. $g_t$ is uniformly continuous
by {prf:ref}`res-la-ns-scaling-is-continuous`.
1. Then, $g_r$ is also uniformly continuous
by {prf:ref}`res-la-ns-scaling-is-continuous`.
1. Note that
$$
g_r(g_t)(\bx) = g_t(g_r(\bx)) = \bx \Forall \bx \in \VV.
$$
1. Thus, $g_t$ is bijective and $g_r$ is its inverse.
1. Since $g_t$ is bijective, $g_t$ is continuous,
and its inverse $g_r$ is continuous,
hence $g_t$ is a {prf:ref}`homeomorphism `.
1. Morever, $g_r$ is also a homeomorphism.
By {prf:ref}`res-ms-homeomorphism-clopen-map`,
$g_t$ and $g_r$ are both a
closed mapping (mapping closed sets to closed sets)
and an open mapping (mapping open sets to open sets).
Consequently,
1. $A$ is open if and only if $g_t(A)$ is open.
1. $A$ is closed if and only if $g_t(A)$ is closed.
Due to {prf:ref}`res-ms-compact-homeomorphism-pres`
(homeomorphisms preserve compact sets)
$A$ is compact if and only if $g_t(A)$ is compact.
Due to {prf:ref}`res-ms-homeomorphism-interior`,
homeomorphisms preserve interiors.
Hence, $A$ has an empty interior
if and only if $g_t(A)$ has an empty interior.
```
## Open Sets
```{prf:theorem} Sum of an open set with any set is open
:label: res-la-sum-open-sets
Let $\VV$ be a normed linear space. Let $A, B \subseteq \VV$.
If $A$ is open, then the sum $A+B$ is open.
```
```{prf:proof}
We proceed as follows:
1. Let $x \in B$.
1. Then the set $x + A$ is open since $A$ is open
and translations preserve open sets
({prf:ref}`res-la-ns-translation-preserve-topology`).
1. Then,
$$
A + B = \bigcup_{x \in B} x + A
$$
is a union of open sets. Hence, it is open.
```
## Closed Sets
```{prf:theorem}
:label: res-la-sum-closed-compact
If $A$ is closed and $B$ is compact, then their sum $A+B$ is closed.
```
```{prf:proof}
Let $\{z_n\}$ with $z_n = a_n + b_n \in A + B$ be a convergent sequence of $A+B$
where $a_n \in A$ and $b_n \in B$.
1. $\{a_n \}$ is a sequence of $A$.
1. $\{b_n \}$ is a sequence of $B$.
1. Let $\lim z_n = z$.
1. Since $B$ is compact, $\{b_n\}$ has a convergent subsequence,
say $\{b_{k_n}\}$ ({prf:ref}`def-ms-compact-characterization`).
1. Let $\lim b_{k_n} = l \in B$.
1. Now consider the sequence $\{a_{k_n}\}$ given by $a_{k_n} = z_{k_n} - b_{k_n}$.
1. $\{z_{k_n}\}$ is convergent since it is a subsequence of a convergent sequence
({prf:ref}`res-ms-subsequence-convergence`).
1. Since $\{z_{k_n}\}$ and $\{b_{k_n}\}$ are both convergent,
hence $\{a_{k_n}\}$ is also convergent.
1. Let $\lim a_{k_n} = m$.
1. Since $A$ is closed. Hence $m \in A$
({prf:ref}`res-ms-closure-convergence`).
1. Now, $\lim a_{k_n} = \lim z_{k_n} - \lim b_{k_n}$ gives us $m = z - l$.
1. Thus, $z = m + l$.
1. But $m \in A$ and $l \in B$.
1. Hence, $z \in A + B$.
1. Thus, every convergent sequence of $A + B$ converges in $A+B$.
1. Thus, $A+B$ is closed. ({prf:ref}`res-ms-closure-convergence`).
```
## Boundedness
By definition $\| \bx \| \geq 0$. Thus, for any
subset $A \subseteq \VV$, $0$ is a lower
bound for the set $\{ \| \bx \| \ST \bx \in A \}$.
If there is an upper bound also for the set of norms,
then the set $A$ is called bounded.
```{index} Normed linear space; bounded set
```
```{prf:definition} Bounded set
:label: def-la-ns-bounded-set
Let $\VV$ be a normed linear space.
A subset $A$ of $\VV$ is called *norm bounded* or simply *bounded*
if there exist $M > 0$ such that:
$$
\| \bx \| \leq M \Forall \bx \in A.
$$
```
Compare the definition with the definition of
{prf:ref}`bounded sets ` in metric spaces.
We can characterize the case where $0$ is indeed
the infimum or greatest lower bound for the set
$\{ \| \bx \| \ST \bx \in A \}$.
```{prf:theorem} Zero as the greatest lower bound
:label: res-la-set-zero-infimum
Let $\VV$ be a normed linear space.
Let $A \subseteq \VV$.
Then, $\inf \{ \| \bx \| \ST \bx \in A \} = 0$
if and only if $\bzero \in \closure A$.
```
```{prf:proof}
Note that $B(\bzero, r)$ is an open ball of radius
$r$ around $\bzero$ given by:
$$
B(\bzero, r) = \{ \bx \in \VV \ST \| \bx \| < r \}.
$$
Suppose $\inf \{ \| \bx \| \ST \bx \in A \} = 0$.
1. Then, for every $r > 0$, there exists $\bx \in A$
such that $\| \bx \| < r$.
1. Thus, for every $r > 0$, $\bx \in B(\bzero, r) \cap A$.
1. Thus, for every $r > 0$, $B(\bzero, r) \cap A \neq \EmptySet$.
1. Thus, $\bzero \in \closure A$.
Now suppose that $\bzero \in \closure A$.
1. Then, for every $r> 0$, $B(\bzero, r) \cap A \neq \EmptySet$.
1. Thus, for every $r > 0$, there exists $\bx \in A$
such that $\| \bx \| < r$.
1. Thus, $\inf \{ \| \bx \| \ST \bx \in A \} = 0$.
```
## Sequences
```{index} Normed linear space; convergence
```
```{prf:definition} Convergence in norm
:label: def-la-ns-convergence-norm
A {prf:ref}`sequence `
$\{ \bx_n \}$ of a normed space $\VV$ is said to
*converge in norm* to $\bx \in \VV$ if
$$
\lim_{n \to \infty} \| \bx - \bx_n \| = 0;
$$
i.e., if $\{\bx_n \}$ converges to $\bx$ with respect to the
metric induced by the norm.
We write this as:
$$
\lim_{n \to \infty} \bx_n = \bx.
$$
```
Following is an alternative proof for
{prf:ref}`res-la-set-zero-infimum` in terms
of convergent sequences.
```{prf:proof}
Assume that $\bzero \in \closure A$.
1. Then, there exists a sequence $\{ \bx_n \}$ such that
$\lim \| \bx_n - \bzero \| = 0$.
1. Thus, $\lim \| \bx_n \| = 0$.
1. Thus, for every $r > 0$, there exists $n_0 \in \Nat$
such that $\| \bx_n \| < r$ for all $n > n_0$.
1. Thus, for every $r > 0$, there exists $\bx \in A$
such that $\| \bx \| < r$.
1. Thus, $\inf \{ \| \bx \| \ST \bx \in A \} = 0$.
Assume that $\inf \{ \| \bx \| \ST \bx \in A \} = 0$.
1. Thus, for every $r > 0$, there exists $\bx \in A$
such that $\| \bx \| < r$.
1. For every $n \in \Nat$, pick a $\bx_n \in A$
such that $\| \bx_n \| < \frac{1}{n}$.
1. Form the sequence $\{ \bx_n \}$.
1. Then, for every $r > 0$, there exists $n_0 \in \Nat$
such that for all $n > n_0$,
$\| \bx_n \| = \| \bx_n - \bzero \| < r$.
1. Thus, $\lim \bx_n = \bzero$.
1. Thus, $\bzero \in \closure A$.
```
## The Calculus of Limits
Let $\{ \bx_n \}$ and $\{ \by_n \}$ be convergent sequences of $\VV$.
Our concern here is to understand what happens to the limits if
the sequences are combined.
Our presentation here is similar to
the presentation for sequences of real numbers
in {ref}`sec:bra:sequences:calculus:limits`.
Let $\lim \{\bx_n\} = \bx$ and $\lim \{\by_n\} = \by$. Then:
```{prf:theorem} Scaling a sequence
:label: res-la-ns-seq-calculus-scaling
$$
\lim \{\alpha \bx_n \} = \alpha \bx \Forall \alpha \in \FF.
$$
```
```{prf:proof}
If $\alpha = 0$, then we have a constant sequence and the result is trivial.
So assume that $\alpha \neq 0$. Then:
$$
\|\alpha \bx_n - \alpha \bx \| = | \alpha | \| \bx_n - \bx \|.
$$
Let $\epsilon > 0$ and choose $n_0 \in \Nat$
such that $\| \bx - \bx_n \| < \frac{\epsilon}{ | \alpha | }$
for all $n > n_0$. Then
$$
\|\alpha \bx_n - \alpha \bx \| = | \alpha | \| \bx_n - \bx \|
< | \alpha | \frac{\epsilon}{ | \alpha | } = \epsilon \Forall n > n_0.
$$
```
```{prf:corollary} Negating a sequence
:label: res-la-ns-seq-calculus-negation
$$
\lim \{-\bx_n \} = -\bx.
$$
```
We get this result by choosing $\alpha = -1$.
```{prf:theorem} Addition of sequences
:label: res-la-ns-seq-calculus-addition
$$
\lim \{\bx_n + \by_n\} = \bx + \by.
$$
```
```{prf:proof}
From triangle inequality we get:
$$
\| \bx_n + \by_n - (\bx + \by) | \leq \| \bx_n - \bx \| + \| \by_n - \by \|.
$$
For any $\epsilon > 0$, choose $n_1$ such that
$\| \bx_n - \bx \| < \frac{\epsilon}{2} \Forall n > n_1$.
Similarly, choose $n_2$ such that
$\| \by_n - \by \| < \frac{\epsilon}{2} \Forall n > n_2$.
Now choose $n_0 = \max (n_1, n_2)$. Then:
$$
\| \bx_n + \by_n - (\bx + \by) \|
\leq \| \bx_n - \bx \| + \| \by_n - \by \|
< \frac{\epsilon}{2} + \frac{\epsilon}{2} = \epsilon
\Forall n > n_0.
$$
```
```{prf:corollary} Subtraction of sequences
:label: res-la-ns-seq-calculus-subtraction
$$
\lim \{\bx_n - \by_n\} = \bx - \by.
$$
```
Negate $\{ \by_n \}$ and add to $\{\bx_n \}$.
## Cauchy Sequences
```{prf:theorem} Cauchy sequence is bounded
:label: res-la-ns-cauchy-seq-bounded
Every {prf:ref}`Cauchy sequence `
in a normed space is bounded.
```
The proof is very similar to the proof for the
boundedness of Cauchy sequences in real line
({prf:ref}`res-bra-cauchy-sequence-bounded`).
```{prf:proof}
Let $\{ \bx_n \}$ be a sequence of a normed space $\VV$.
1. Choose $\epsilon = 1$.
1. Then there exists $n_0 \in \Nat$
such that $\| \bx_n - \bx_m \| < 1$ whenever $m, n \geq n_0$.
1. In particular, the statement is valid when $m = n_0$.
i.e. $\| \bx_n - \bx_{n_0} \| < 1$.
1. But,
$$
\begin{aligned}
& \| \bx_n - \bx_{n_0} \| < 1\\
& \implies | \| \bx_n \| - \| \bx_{n_0 } \| | < 1\\
& \implies \|\bx_n \| < 1 + \| \bx_{n_0 } \| \Forall n \geq n_0.
\end{aligned}
$$
1. Choosing $M = \max(\|x_1\|, \dots, \|\bx_{n_0-1}\|, \|\bx_{n_0}\| + 1)$,
it is clear that $\| x_n \| \leq M$
1. Hence $\{ \bx_n \}$ is bounded.
```
## Linear Transformations
### Boundedness
```{index} Linear transformation; bounded
```
````{prf:definition} Bounded linear transformation
:label: def-la-bounded-lin-map
Let $(\VV, \| \cdot \|_v)$ and $(\WW, \| \cdot \|_w)$ be
normed linear spaces. Let
$T : \VV \to \WW$ be a linear transformation.
We say that $T$ is *bounded* (in the sense of linear transformations)
if the set
```{math}
:label: eq-la-ns-bounded-func
S \triangleq \left \{ \frac{\| T(\bx) \|_w}{\| \bx \|_v} \ST \bx \in \VV, \bx \neq \bzero \right \}
```
is bounded.
````
Note that the set $S$ in {eq}`eq-la-ns-bounded-func`
is trivially bounded from below by $0$.
Thus, by bounded we mean, bounded from above.
The notion of boundedness for linear transformations
is different from the notion of boundedness for
bounded functions in general metric spaces
as defined in {prf:ref}`def-ms-bounded-function`.
There, we posit that the range of the function is bounded.
For bounded linear transformations, the range (in norm) may be unbounded,
yet the ratio of the norms of output and input is bounded.
Since linear transformations commute with scalar multiplication
as in $T(t \bx) = t T(\bx)$, hence the norm of the output is
unbounded unless $T$ is an identically $\bzero$ linear transformation.
```{div}
If there exists $\bv \in \VV$ such that $T(\bv) \neq \bzero$, then
$$
\| T(t \bv)\|_w = | t| \| T (\bv) \|_w \to \infty
\text{ as } | t | \to \infty.
$$
Thus, the idea of a bounded range is not very
useful for linear transformations.
```
````{prf:remark} Bounded linear transformation in terms of unit norm vectors
:label: res-la-ns-bounded-lin-func-unit-norm-vecs
For $T, \bx$ as in {eq}`eq-la-ns-bounded-func`, and any nonzero $t \in \FF$,
we have
$$
\frac{\| T( t \bx) \|_w}{\| t \bx \|_v}
= \frac{\|t T( \bx) \|_w}{ |t | \| \bx \|_v}
= \frac{| t | \| T( \bx) \|_w}{ |t | \| \bx \|_v}
= \frac{\| T( \bx) \|_w}{ | | \| \bx \|_v}.
$$
In particular, if we choose $t = \frac{1}{\|\bx \|_v}$, then
$\| t \bx \|_v = 1$.
Hence, every element in set $S$ in {eq}`eq-la-ns-bounded-func`
equals $\| T (\bx) \|_w$ for some unit vector $\bx$.
It follows that
```{math}
:label: eq-la-ns-bounded-func-2
S = \left \{ \frac{\| T(\bx) \|_w}{\| \bx \|_v}
\ST \bx \in \VV, \bx \neq \bzero \right \}
= \left \{ \| T(\bx) \|_w
\ST \bx \in \VV, \| \bx \|_v = 1 \right \}.
```
Some authors prefer this description of $S$ as the
definition of bounded linear transformations.
````
```{prf:example} Unbounded linear transformation
:label: ex-la-ns-unbounded-transformation
It is possible to construct unbounded linear transformations
in infinite dimensional normed linear spaces.
We shall consider a {prf:ref}`sequence space `
as described below.
Let $V$ be the set of all real sequences with
finitely many nonzero terms.
$$
V = \{ \{ x_n \} \ST \exists m \in \Nat \text{ with }
x_n = 0 \Forall n \geq m \}.
$$
It is easy to verify that $V$ is a linear subspace
of the {prf:ref}`space of all sequences `
$\RR^{\infty}$.
For any $\bx \in V$ given by $\bx = \{ x_n \}$,
define the supremum norm as:
$$
\| \bx \| = \sup \{ |x_1 |, |x_2 |, \dots \}.
$$
It is easy to verify that it is indeed a norm.
Define a transformation $T : V \to V$ as
$$
T(x_1, x_2, x_3, \dots, x_n, 0, 0, \dots) =
(x_1, 2 x_2, 3 x_3, \dots, n x_n, 0, 0, \dots).
$$
It is easy to verify that $T$ is linear.
For any sequence $\be_n = (0, 0, \dots, 1, 0, 0, \dots)$
with $1$ in n-th position and all other entries 0,
$\| \be_n \| = 1$ and $ \| T (\be_n) \| = n$.
Since we can construct $\be_n$ for every $n \in \Nat$, hence
the set $S$ as defined in {eq}`eq-la-ns-bounded-func-2`
is unbounded.
Consequently, $T$ is an unbounded linear transformation.
```
### Continuity
```{prf:theorem} Characterization of continuity for linear transformations
:label: res-la-ns-continuity-lin-map
Let $(\VV, \| \cdot \|_v)$ and $(\WW, \| \cdot \|_w)$ be
normed linear spaces. Let
$T : \VV \to \WW$ be a linear transformation.
Then, the following statements are equivalent.
1. $T$ is continuous at one point in $\VV$.
1. $T$ is continuous at $\bzero \in \VV$.
1. $T$ is bounded.
1. $T$ is Lipschitz continuous at $\bzero \in \VV$.
1. $T$ is Lipschitz continuous.
1. $T$ is uniformly continuous.
1. $T$ is continuous.
```
```{prf:proof}
(1) $\implies$ (2)
Assume $T$ is continuous at $\bx_0 \in \VV$.
1. Let $\epsilon > 0$ and let $\delta > 0$ such that
if $\| \bx - \bx_0 \|_v < \delta$ then $ \| T(\bx) - T(\bx_0) \|_w < \epsilon$.
1. Then, for all $\by \in B_v(\bzero, \delta)$, we have
$$
\| (\by + \bx_0) - \bx_0 \|_v = \| \by \|_v < \delta.
$$
1. Hence, $\| T(\by + \bx_0) - T(\bx_0) \|_w < \epsilon$.
1. But $T$ is linear. Hence
$$
T(\by + \bx_0) - T(\bx_0)
= T(\by) = T(\by) - \bzero = T(\by) - T(\bzero).
$$
1. Thus, $\| T(\by) - T(\bzero) \|_w < \epsilon$.
1. Thus, for any $\epsilon > 0$, there exists $\delta > 0$ such that
$\| \by - \bzero \|_v < \delta$ implies $\| T (\by) - T(\bzero) \|_w < \epsilon$.
1. Thus, $T$ is continuous at $\bzero \in \VV$.
(2) $\implies$ (3)
Assume that $T$ is continuous at $\bzero \in \VV$.
1. Then, for any $\by \in \VV$,
$$
\| T(\by) - T(\bzero) \|_w = \| T (\by) - \bzero \|_w
= \| T (\by) \|_w.
$$
1. Let $\epsilon = 1$ and choose $\delta > 0$ such that
if $\| \by \|_v < \delta$ then
$\| T (\by) \|_w < 1$.
We can choose such $\delta > 0$ since $T$ is
continuous at $\bzero$.
1. Let $\bx \in \VV$ be any nonzero vector.
1. Let $\by = \frac{\delta}{2 \| \bx \|_v } \bx$.
1. Then, $\| \by \|_v = \frac{\delta}{2} < \delta$.
1. And $\bx = \frac{2 \| \bx \|_v}{\delta} \by$.
1. Then,
$$
\| T(\bx)\|_w
&= \left \| T \left ( \frac{2 \| \bx \|_v}{\delta} \by \right ) \right \|_w\\
&= \left \| \frac{2 \| \bx \|_v}{\delta} T ( \by ) \right \|_w\\
&= \frac{2 \| \bx \|_v}{\delta} \| T (\by) \|_w \\
&< \frac{2 \| \bx \|_v}{\delta} \times 1 \\
&= \frac{2 }{\delta} \| \bx \|_v.
$$
1. Thus,
$$
\frac{\| T(\bx)\|_w }{\| \bx \|_v} < \frac{2 }{\delta}
$$
holds true for every nonzero $\bx \in \VV$.
1. Thus, the set in {eq}`eq-la-ns-bounded-func`
is bounded from above by $\frac{2 }{\delta}$.
1. Thus, $T$ is bounded.
(3) $\implies$ (4)
Assume that $T$ is bounded.
Let the set $S$ in {eq}`eq-la-ns-bounded-func`
have an upper bound $M > 0$.
Let $\bx \in \VV$ be a nonzero vector.
Then,
$$
\| T (\bx) - T(\bzero) \|_w
&= \| T (\bx) - \bzero \|_w \\
&= \| T (\bx) \|_w \\
&= \left \| T \left ( \| \bx \|_v \frac{\bx}{\| \bx \|_v} \right) \right \|_w\\
&= \| \bx \|_v \left \| T \left ( \frac{\bx}{\| \bx \|_v} \right) \right \|_w\\
&\leq \| \bx \|_v M\\
&= M \| \bx - \bzero \|_v.
$$
Thus, there exists $M > 0$ such that for every nonzero $\bv \in \VV$,
$$
\| T (\bx) - T(\bzero) \|_w \leq M \| \bx - \bzero \|_v.
$$
Thus, $T$ is Lipschitz continuous at $\bzero \in \VV$.
(4) $\implies$ (5)
We assume that $T$ is Lipschitz continuous at $\bzero \in \VV$.
Then, let $K, \delta > 0$ such that for every $\by \in \VV$
with $\| \by \|_v < \delta$, we have
$$
\| T (\by) \|_w \leq K \| \by \|_v.
$$
1. Let $\bx \in \VV$ be a nonzero vector.
1. Let $\by = \frac{\delta}{2} \frac{\bx}{\| \bx \|_v}$.
1. Then, $\| \by \|_v = \frac{\delta}{2} < \delta$.
1. Hence, $\| T (\by) \|_w \leq K \| \by \|_v$.
1. Now, using linearity of $T$:
$$
\| T(\bx)\|_w
&= \left \| T \left ( \frac{2 \| \bx \|_v}{\delta} \by \right ) \right \|_w\\
&= \left \| \frac{2 \| \bx \|_v}{\delta} T ( \by ) \right \|_w\\
&= \frac{2 \| \bx \|_v}{\delta} \| T (\by) \|_w \\
&\leq \frac{2 \| \bx \|_v}{\delta} K \| \by \|_v \\
&= K \| \bx \|_v.
$$
1. Thus, we have $\| T(\bx)\|_w \leq K \| \bx \|_v$
for every nonzero $\bx \in \VV$.
1. Now, let $\bx, \bz \in \VV$ be distinct.
1. Using linearity of $T$
$$
\| T (\bx) - T(\bz) \|_w
&= \| T (\bx - \bz) \|_w\\
\leq K \| \bx - \bz \|_v.
$$
1. Hence, $T$ is Lipschitz continuous.
(5) $\implies$ (6)
Assume that $T$ is Lipschitz continuous.
Let $K >0$ such that for every $\bx, \by \in \VV$,
$$
\| T(\bx) - T(\by)\|_w \leq K \| \bx - \by \|_v.
$$
1. Let $\epsilon > 0$.
1. Choose $\delta = \frac{\epsilon}{K}$.
1. Then, for every $\bx, \by \in \VV$ with
$\| \bx - \by \|_v < \delta$, we have
$$
\| T(\bx) - T(\by)\|_w
\leq K \| \bx - \by \|_v
< K \delta = K \frac{\epsilon}{K} = \epsilon.
$$
1. Hence, $T$ is uniformly continuous.
(6) $\implies$ (7)
A uniformly continuous function is trivially continuous.
(7) $\implies$ (1)
A continuous function is trivially continuous at a point.
```
## Equivalent Norms
```{index} Equivalent norms
```
```{prf:definition} Equivalent norms
:label: def-la-ns-equivalent-norms
Let $\VV$ be a vector space.
Let $\| \cdot \|_a : \VV \to \RR$ and $\| \cdot \|_b : \VV \to \RR$
be two different norms defined on $\VV$.
We say that the two norms are *equivalent* if
there exist constants $c_1, c_2 > 0$ such that
for every $\bv \in \VV$, we have
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
```
Note that the zero-dimensional vector space $\{\bzero \}$
has only one norm which is identically zero. Thus,
the concept of equivalent norms is not interesting for it.
Thus, we shall concern ourselves only with vector spaces
of dimension greater than zero for the purposes of
norm equivalence.
There are several ways to express the equivalent norm
inequalities.
```{prf:theorem} Characterization of equivalent norms
:label: res-la-ns-equivalent-norms-charact
Let $\VV$ be a vector space.
Let $\| \cdot \|_a : \VV \to \RR$ and $\| \cdot \|_b : \VV \to \RR$
be two different norms defined on $\VV$.
The following statements are equivalent.
1. The norms $\| \cdot \|_a$ and $\| \cdot \|_b$ are equivalent.
1. There exist constants $c_1, c_2 > 0$ such that for every $\bv \in \VV$
$$
c_1 \| \bv \|_b \leq \| \bv \|_a \leq c_2 \| \bv \|_b.
$$
1. There exists $c > 0$ such that for all $\bv \in \VV$
$$
\frac{1}{c} \| \bv \|_b \leq \| \bv \|_a \leq c \| \bv \|_b.
$$
```
```{prf:proof}
(1) $\implies$ (2)
1. By hypothesis, the norms
$\| \cdot \|_a$ and $\| \cdot \|_b$ are equivalent.
1. Thus, there exist $d_1, d_2 > 0$ such that for every $\bv \in \VV$
$$
\| \bv \|_a \leq d_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq d_2 \| \bv \|_a.
$$
1. Let $c_1 = \frac{1}{d_2} > 0$ and $c_2 = d_1 > 0$.
1. The inequalities become:
$$
\| \bv \|_a \leq c_2 \| \bv \|_b
\text{ and }
c_1 \| \bv \|_b \leq \| \bv \|_a.
$$
1. Combining, we get:
$$
c_1 \| \bv \|_b \leq \| \bv \|_a \leq c_2 \| \bv \|_b.
$$
(2) $\implies$ (3)
1. By hypothesis, there exist constants $c_1, c_2 > 0$
such that for every $\bv \in \VV$
$$
c_1 \| \bv \|_b \leq \| \bv \|_a \leq c_2 \| \bv \|_b.
$$
1. Let $c = \max \left \{ \frac{1}{c_1}, c_2 \right \}$.
Note that $c > 0$ since $c_1, c_2 > 0$.
1. Then, $c \geq c_2$.
1. Thus, $\| \bv \|_a \leq c_2 \| \bv \|_b \leq c \| \bv \|_b$ for every $\bv \in \VV$.
1. Also, $c \geq \frac{1}{c_1} \implies \frac{1}{c} \leq c_1$.
1. Thus, $\frac{1}{c} \| \bv \|_b \leq c_1 \| \bv \|_b \leq \| \bv \|_a$
for every $\bv \in \VV$.
1. Combining, we get:
$$
\frac{1}{c} \| \bv \|_b \leq \| \bv \|_a \leq c \| \bv \|_b.
$$
(3) $\implies$ (1)
1. By hypothesis, there exists $c > 0$ such that for all $\bv \in \VV$
$$
\frac{1}{c} \| \bv \|_b \leq \| \bv \|_a \leq c \| \bv \|_b.
$$
1. Let $c_1 = c$ and $c_2 = c$.
1. Thus, $\| \bv \|_a \leq c_1 \| \bv \|_b$ for all $\bv \in \VV$.
1. And $\frac{1}{c_2} \| \bv \|_b \leq \| \bv \|_a$
implies $\| \bv \|_b \leq c_2 \| \bv \|_a$ for all $\bv \in \VV$.
1. Thus, there exist constants $c_1, c_2 > 0$ given by $c_1 = c_2 = c$
such that for every $\bv \in \VV$, we have
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
1. Thus, the two norms are equivalent.
```
```{prf:theorem} Norm equivalence = Identical bounded sets
:label: res-la-ns-norm-eq-same-bounded
Let $\VV$ be a vector space.
Let $\| \cdot \|_a : \VV \to \RR$ and $\| \cdot \|_b : \VV \to \RR$
be two different norms defined on $\VV$.
The following statements are equivalent.
1. The norms $\| \cdot \|_a$ and $\| \cdot \|_b$ are equivalent.
1. Every set $A \subseteq \VV$ is bounded in
$(\VV, \| \cdot \|_a)$ if and only if
it is bounded in $(\VV, \| \cdot \|_b)$.
```
```{prf:proof}
(1) $\implies$ (2)
1. By hypothesis, the norms
$\| \cdot \|_a$ and $\| \cdot \|_b$ are equivalent.
1. Thus, there exist $c_1, c_2 > 0$ such that for every $\bv \in \VV$
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
1. Let $A$ be bounded in $(\VV, \| \cdot \|_a)$.
1. Then, there exists a constant $M > 0$ such that
$$
\| \bx \|_a \leq M \Forall \bx \in A.
$$
1. But then,
$$
\| \bx \|_b \leq c_2 \| \bx \|_a \leq M c_2
\Forall \bx \in A.
$$
1. Thus, $A$ is bounded in $(\VV, \| \cdot \|_b)$.
1. A similar reasoning shows that
if $A$ is bounded in $(\VV, \| \cdot \|_b)$,
then $A$ must be bounded in
$(\VV, \| \cdot \|_a)$ also.
(2) $\implies$ (1)
1. By hypothesis, every set $A \subseteq \VV$ is bounded in
$(\VV, \| \cdot \|_a)$ if and only if
it is bounded in $(\VV, \| \cdot \|_b)$.
1. For contradiction, assume that the norms are not equivalent
and there is no constant $c_1 > 0$ such that
$\| \bv \|_a \leq c_1 \| \bv \|_b$ for every $\bv \in \VV$.
1. Then, for each $n \in \Nat$, there exists $\bx_n \in \VV$
such that $\| \bx_n \|_a > n \| \bx_n \|_b$.
1. In particular, $\bx_n \neq \bzero$.
1. The set $S = \{ \frac{\bx_n}{\| \bx_n \|_b} | n \in \Nat \}$
is bounded in $(\VV, \| \cdot \|_b)$
since each point is unit norm.
1. Then, $S$ is bounded in $(\VV, \| \cdot \|_a)$ also
by hypothesis.
1. Thus, $\left \| \frac{\bx_n}{\| \bx_n \|_b} \right \|_a \leq C$
for some $C > 0$ and every $n \in \Nat$.
1. It implies $\| \bx_n \|_a \leq C \| \bx_n \|_b$ for every
$n \in \Nat$.
1. But, this contradictions with our choice above
as $\| \bx_n \|_a > n \| \bx_n \|_b$.
1. Thus, there must exist a constant $c_1 > 0$ such that
$\| \bv \|_a \leq c_1 \| \bv \|_b$ for every $\bv \in \VV$.
1. A similar reasoning shows that there must exist a constant $c_2 > 0$
such that
$\| \bv \|_b \leq c_2 \| \bv \|_a$ for every $\bv \in \VV$.
1. Thus, the two norms must be equivalent.
```
```{prf:theorem}
:label: res-la-ns-norm-equivalence-rel
Equivalence of norms is an equivalence relation
on the set of norms for a given vector space.
```
```{prf:proof}
Let $\VV$ be a given vector space.
Let $\| \cdot \|_a : \VV \to \RR$,
$\| \cdot \|_b : \VV \to \RR$,
$\| \cdot \|_c : \VV \to \RR$,
be norms defined on $\VV$.
We shall say that $\| \cdot \|_a \sim \| \cdot \|_b$
if the norms $\| \cdot \|_a$ and $\| \cdot \|_b$
are equivalent; i.e.,
there exist constants $c_1, c_2 > 0$
such that for every $\bv \in \VV$, we have
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
[Reflexivity]
1. Choose $c_1 = c_2 = 1$. Then,
$$
\| \bv \|_a \leq \| \bv \|_a
$$
for every $\bv \in \VV$.
1. Hence, $\| \cdot \|_a \sim \| \cdot \|_a$.
[Symmetry]
1. Let $\| \cdot \|_a \sim \| \cdot \|_b$.
1. Then, there exist constants $c_1, c_2 > 0$
such that for every $\bv \in \VV$, we have
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
1. Choose $d_1 = c_2$ and $d_2 = c_1$.
1. Then, for every $\bv \in \VV$, we have
$$
\| \bv \|_b \leq d_1 \| \bv \|_a
\text{ and }
\| \bv \|_a \leq d_2 \| \bv \|_b.
$$
1. Thus, $\| \cdot \|_b \sim \| \cdot \|_a$.
[Transitivity]
1. Let $\| \cdot \|_a \sim \| \cdot \|_b$ and
$\| \cdot \|_b \sim \| \cdot \|_c$.
1. Then, there exist constants $c_1, c_2 > 0$
such that for every $\bv \in \VV$, we have
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
1. And, there exist constants $d_1, d_2 > 0$
such that for every $\bv \in \VV$, we have
$$
\| \bv \|_b \leq d_1 \| \bv \|_c
\text{ and }
\| \bv \|_c \leq d_2 \| \bv \|_b.
$$
1. Let $e_1 = c_1 d_1 > 0$ and $e_2 = c_2 d_2 > 0$.
1. Then, for every $\bv \in \VV$:
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\leq c_1 (d_1 \| \bv \|_c)
= (c_1 d_1)\| \bv \|_c = e_1 \| \bv \|_c.
$$
1. Similarly, for every $\bv \in \VV$:
$$
\| \bv \|_c \leq d_2 \| \bv \|_b
\leq d_2 (c_2 \| \bv \|_a)
= (d_2 c_2) \| \bv \|_a
= e_2 \| \bv \|_a.
$$
1. Thus, $\| \cdot \|_a \sim \| \cdot \|_c$.
```
```{prf:theorem} Norm equivance $\implies$ metric strong equivalence
:label: res-la-ns-norm-eq-metric-eq
If two norms are equivalent, then there associated
metrics are
{prf:ref}`strongly equivalent `.
```
```{prf:proof}
Let $\VV$ be a given vector space.
Let $\| \cdot \|_a : \VV \to \RR$,
and $\| \cdot \|_b : \VV \to \RR$
be norms defined on $\VV$ which are equivalent.
Then, there exist constants $c_1, c_2 > 0$
such that for every $\bv \in \VV$, we have
$$
\| \bv \|_a \leq c_1 \| \bv \|_b
\text{ and }
\| \bv \|_b \leq c_2 \| \bv \|_a.
$$
The associated metrics are given by
$$
d_a(\bx, \by) = \| \bx - \by \|_a
\text{ and }
d_b(\bx, \by) = \| \bx - \by \|_b.
$$
Clearly, for every $\bx, \by \in \VV$:
$$
d_a(\bx, \by) = \| \bx - \by \|_a \leq c_1 \| \bx - \by \|_b
= c_1 d_b(\bx, \by).
$$
Similarly,
$$
d_b(\bx, \by) = \| \bx - \by \|_b \leq c_2 \| \bx - \by \|_a
= c_2 d_a(\bx, \by).
$$
Thus, there exist $c_1, c_2 > 0$ such that
$$
d_a(\bx, \by) \leq c_1 d_b(\bx, \by)
\text{ and }
d_b(\bx, \by) \leq c_2 d_a(\bx, \by)
\Forall \bx, \by \in \bx, \by \in \VV.
$$
Thus, the two associated metrics are
{prf:ref}`strongly equivalent `.
```
### Norms on Finite Dimensional Spaces
```{prf:theorem} Equivalence of norms on a finite dimensional vector space
:label: res-la-ns-finite-all-norms-eq
Let $\VV$ be a finite dimensional vector space
over the scalar field $\FF$ where $\FF$ is $\RR$ or $\CC$.
Then, all norms on $\VV$ are equivalent.
```
Reaching this conclusion requires significant amount
of work on the norms on Euclidean spaces
which are discussed in detail in
{ref}`sec:la:real-euclidean-space`.
Readers are advised to read the material on
norms on $\RR^n$ and the fact that
all norms on $\RR^n$ are equivalent
before proceeding further.
```{prf:proof}
If $0 = \dim \VV$, then there is only one norm
and there is nothing to prove.
So assume that $\dim \VV > 0$.
Then, $\VV$ is isomorphic to $\RR^n$ for some $n$.
1. Let $L : \RR^n \to \VV$ be an isomorphism.
1. Now, if $\| \cdot \|$ is a norm on $\VV$, then
the function $\| \cdot \|_L : \RR^n \to \RR$
defined by
$$
\| \bx \|_L = \| L (\bx)\|
$$
is a norm on $\RR^n$.
1. Let $\| \cdot \|_a$ and $\| \cdot \|_b$ be
two different norms on $\VV$.
1. Let $\| \cdot \|_{L,a}$ and $\| \cdot \|_{L,b}$
be the corresponding induced norms on $\RR^n$.
1. By {prf:ref}`res-la-ns-norms-euclidean-eq`,
all norms on $\RR^n$ are equivalent.
1. Hence, $\| \cdot \|_{L,a}$ and $\| \cdot \|_{L,b}$
are equivalent.
1. Hence, there exist constants $c_1, c_2 > 0$ such that
$$
\| \bx \|_{L, a} \leq c_1 \| \bx \|_{L, b}
\text{ and }
\| \bx \|_{L, b} \leq c_2 \| \bx \|_{L, a}
$$
holds true for every $\bx \in \RR^n$.
1. Then, for every $\bv \in \VV$
$$
\| \bv \|_a &= \| L (L^{-1} (\bv)) \|_a\\
&= \| L^{-1} (\bv) \|_{L, a} \\
&\leq c_1 \| L^{-1} (\bv) \|_{L, b}\\
&= c_1 \| L (L^{-1} (\bv)) \|_b \\
&= c_1 \| \bv \|_b.
$$
1. Similarly, for every $\bv \in \VV$
$$
\| \bv \|_b \leq c_2 \| \bv \|_a
$$
holds true.
1. Thus, the two norms are equivalent.
Since, the two norms chosen were arbitrary,
hence all norms on $\VV$ are equivalent.
```
```{index} Norm topology
```
```{prf:definition} Norm topology
:label: def-la-ns-finite-norm-topology
Let $\VV$ be a finite dimensional space.
Since all norms on $\VV$ are equivalent,
hence, they induce the same topology
(family of open sets, closed sets, compact sets).
The topology induced by a norm (the collection of open sets)
on a finite dimensional space
is called its *norm topology*.
```
### Continuity
Continuity of a function doesn't depend
on the choice of a norm within the class
of all equivalent norms.
In other words, if a function $f$ is
continuous for a given norm, then it
is continuous for all norms equivalent to it.
```{prf:theorem} Continuity with equivalent norms
:label: res-la-ns-cont-eq-norms
Let $\VV$ be a given vector space.
Let $\| \cdot \|_a : \VV \to \RR$,
and $\| \cdot \|_b : \VV \to \RR$
be norms defined on $\VV$ which are equivalent.
Let $d_a$ and $d_b$ be corresponding metrics.
Let $(X, d)$ be a metric space.
Let $f : X \to \VV$ be a (total) function.
Then, $f : (X, d) \to (\VV, d_a)$ is continuous
if and only if $f : (X, d) \to (\VV, d_b)$
is continuous.
Similarly, let $g : \VV \to X$ be a (total) function.
Then, $g : (\VV, d_a) \to (X, d)$ is continuous
if and only if $g : (\VV, d_b) \to (X, d)$
is continuous.
```
```{prf:proof}
By {prf:ref}`res-la-ns-norm-eq-metric-eq`,
the metrics $d_a$ and $d_b$ are strongly equivalent.
By {prf:ref}`res-ms-strong-eq-metric-eq`, they are
equivalent.
Thus, they determine the same topology on $\VV$.
Thus, a subset $A \subseteq \VV$ is open
in $(\VV, d_a)$ if and only if it is open in $(\VV, d_b)$.
Now, assume $f : (X, d) \to (\VV, d_a)$ to be continuous.
1. Let $A$ be an open set of $(\VV, d_a)$.
1. Then, $f^{-1}(A)$ is an open set of $(X, d)$
since $f : (X, d) \to (\VV, d_a)$ is continuous
({prf:ref}`res-ms-continuous-function-characterization`).
1. But $A$ is also an open set of $(\VV, d_b)$
since both metrics determine same open sets.
1. Thus, whenever $A$ is an open set of $(\VV, d_b)$,
$f^{-1}(A)$ is an open set of $(X,d)$.
1. Thus, $f : (X, d) \to (\VV, d_b)$ is continuous
due to {prf:ref}`res-ms-continuous-function-characterization`.
A similar reasoning shows that
if $f : (X, d) \to (\VV, d_b)$ is continuous
then $f : (X, d) \to (\VV, d_a)$ must be continuous too.
Combining, $f : (X, d) \to (\VV, d_a)$ is continuous
if and only if $f : (X, d) \to (\VV, d_b)$
is continuous.
Now, assume $g : (\VV, d_a) \to (X, d)$ to be continuous.
1. Let $A$ be an open set of $(X, d)$.
1. Then, $g^{-1}(A)$ is an open set of $(\VV, d_a)$
since $g : (\VV, d_a) \to (X, d)$ is continuous
({prf:ref}`res-ms-continuous-function-characterization`).
1. But $g^{-1}(A)$ is also an open set of $(\VV, d_b)$
since both metrics determine same open sets.
1. Thus, whenever $A$ is an open set of $(X, d)$,
$g^{-1}(A)$ is an open set of $(\VV, d_b)$.
1. Thus, $g : (\VV, d_b) \to (X, d)$ is continuous
due to {prf:ref}`res-ms-continuous-function-characterization`.
A similar reasoning shows that
if $g : (\VV, d_b) \to (X, d)$ is continuous
then $g : (\VV, d_a) \to (X, d)$ must be continuous too.
Combining,
$g : (\VV, d_a) \to (X, d)$ is continuous
if and only if $g : (\VV, d_b) \to (X, d)$
is continuous.
```
```{prf:remark} Continuity in finite dimensional spaces
:label: rem-la-ns-cont-finite-space
In the special case where $\VV$
is finite dimensional, the continuity of $f$
or $g$ is independent of the choice of norm
chosen on $\VV$ since all norms are equivalent.
```
## Linear Transformations in Finite Dimensional Spaces
```{prf:theorem} Linear transformations in finite dimensional spaces are bounded
:label: res-la-ns-finite-bounded-transformation
Let $(\VV, \| \cdot \|_v)$ and $(\WW, \| \cdot \|_w)$ be
normed linear spaces. Let
$T : \VV \to \WW$ be a linear transformation.
If $\VV$ is finite dimensional, then $T$ is
{prf:ref}`bounded `.
```
```{prf:proof}
If $\dim \VV = 0$, then $\VV = \{ \bzero \}$
and any linear transformation is bounded.
Hence, let $\dim \VV > 0$.
Since $\VV$ is finite dimensional, we can choose a basis
$\BBB = \{\be_1, \dots, \be_n \}$ for $\VV$.
1. Let $c_i = \| T (\be_i) \|_w$ for $i=1,\dots,n$
and $c = \max \{ c_1, \dots, c_n \}$.
1. Let $\bx \in \VV$.
1. It can be uniquely written as
$$
\bx = t_1 \be_1 + \dots + t_n \be_n.
$$
1. Then,
$$
\| T (\bx) \|_w
&= \| T (t_1 \be_1 + \dots + t_n \be_n) \|\\
&= \| t_1 T(\be_1) + \dots + t_n T(\be_n) \| \\
&\leq |t_1| \| T (\be_1) \|_w + \dots + |t_n | \| T (\be_n) \|_w \\
&\leq c (|t_1 | + \dots + |t_n |).
$$
1. Note that the function $ \| \cdot \|_* : \VV \to \RR$
given by
$$
\| \bx\|_* = |t_1 | + \dots + |t_n |
$$
is a norm on $\VV$.
1. By {prf:ref}`res-la-ns-finite-all-norms-eq`,
all norms on $\VV$ are equivalent since $\VV$
is finite dimensional.
1. Thus, there exists $c_1 > 0$ such that
$$
\| \bx\|_* \leq c_1 \| \bx\|_v
$$
for every $\bx \in \VV$.
1. Thus, $\| T (\bx) \|_w \leq c c_1 \| \bx\|_v$
for every $\bx \in \VV$.
1. Thus, for every nonzero $\bx \in \VV$,
$$
\frac{\| T (\bx) \|_w}{\| \bx\|_v} \leq c c_1.
$$
1. Thus, the set $S$ in {eq}`eq-la-ns-bounded-func`
is bounded.
1. Thus, $T$ is bounded.
```
```{prf:theorem} Linear transformations in finite dimensional spaces are continuous
:label: res-la-ns-finite-continuous-transformation
Let $(\VV, \| \cdot \|_v)$ and $(\WW, \| \cdot \|_w)$ be
normed linear spaces. Let
$T : \VV \to \WW$ be a linear transformation.
If $\VV$ is finite dimensional, then $T$ is
continuous as well as uniformly continuous
as well as Lipschitz continuous.
```
```{prf:proof}
By {prf:ref}`res-la-ns-finite-bounded-transformation`,
$T$ is bounded.
The rest follows from the characterization of
continuous linear transformations on vector spaces
in {prf:ref}`res-la-ns-continuity-lin-map`.
```
## Linear Subspaces
```{prf:theorem} Linear subspaces are closed
:label: res-la-subspace-closed
Every subspace of a finite dimensional normed linear space
$\VV$ is a closed set.
```
```{prf:proof}
Let $\VV$ be a finite dimensional subspace
equipped with a norm $\| \cdot \|$.
The trivial subspace $\{\bzero \}$ is closed
since it is a singleton. The space $\VV$
is closed by definition.
Let $W$ be a proper nontrivial subspace of $\VV$.
Then, there exists a linear transformation $T$
such that
$$
T (\bx) = \bzero \Forall \bx \in W;
$$
i.e., $W$ is the kernel of $T$.
Then,
$$
W = T^{-1}(\{ \bzero \}).
$$
By {prf:ref}`res-la-ns-finite-continuous-transformation`,
$T$ is continuous.
And the set $\{ \bzero \}$ is a singleton, hence
closed.
Then, by {prf:ref}`res-ms-continuous-function-characterization`
(5), $W = T^{-1}(\{ \bzero \})$ is a closed set.
```
## Completeness and Banach Spaces
```{index} Banach space
```
```{prf:definition} Banach space
:label: def-la-banach-space
A normed space $\VV$ that is
{prf:ref}`complete `
with respect to the metric induced by its norm
is called a *Banach space*.
In other words, $\VV$ is a Banach space if
every Cauchy sequence of $\VV$ converges in $\VV$.
```
```{prf:example} Examples of Banach spaces
The spaces in examples below have been described
in detail elsewhere. Follow the links.
1. The {prf:ref}`Euclidean space `
$\RR^n$ is complete with respect to the
{prf:ref}`Euclidean norm `.
Thus, it is a Banach space.
1. The {prf:ref}`space`
of bounded real valued functions $B(X)$ over a nonempty
set $X$ equipped with
{prf:ref}`sup norm `
is {prf:ref}`complete `.
```
```{prf:theorem} $n$-dim normed linear spaces are complete
:label: res-la-ndim-complete
Let $\VV$ be an $n$-dimensional vector space over $\FF$
where $\FF$ is either $\RR$ or $\CC$.
Let $\VV$ be equipped with a norm $\| \cdot \| : \VV \to \RR$
making it a normed linear space.
Then $\VV$ is
{prf:ref}`complete `.
In other words, every finite dimensional
normed linear space is complete.
```
```{prf:proof}
$\VV$ is a normed linear space with $n = \dim \VV$.
1. Let $\BBB = \{\be_1, \dots, \be_n \}$ be a basis for $\VV$.
1. For any $\bv \in \VV$, we can represent it in the basis
$\BBB$ as
$$
\bv = v_1 \be_1 + \dots + v_n \be_n.
$$
1. We can define the $\| \cdot \|_1$ norm as
$$
\| \bv \|_1 = |v_1| + \dots + | v_n |.
$$
1. By {prf:ref}`res-la-ns-finite-all-norms-eq`, all
norms on $\VV$ are equivalent.
1. Thus, $\| \cdot \|_1$ and $\| \cdot \|$ are equivalent.
1. Thus, there are $c_1, c_2 > 0$ such that for every
$\bv \in \VV$,
$$
c_1 \| \bv \|_1 \leq \| \bv \| \leq c_2 \| \bv \|_1.
$$
1. Let $\{ \bv_k \}$ be a Cauchy sequence of $\VV$.
1. Let $\epsilon > 0$.
1. Since $\{ \bv_k \}$ is a Cauchy sequence of $\VV$,
hence there exists $N$ such that for every
$k, l > N$,
$$
\| \bv_k - \bv_l \| < \epsilon.
$$
1. Then,
$$
\epsilon &> \| \bv_k - \bv_l \| \\
&\geq c_1 \| \bv_k - \bv_l \|_1 \\
&= c_1 \sum_{i=1}^n | v_{k i} - v_{l i} |\\
&\geq c_1 | v_{k i} - v_{l i} |
$$
for every $i \in 1,\dots,n$.
1. Hence $\{ v_{k i} \}$ is a Cauchy sequence of $\FF$ for every $i \in 1,\dots,n$.
1. Since both $\RR$ and $\CC$ are complete, hence
$\{ v_{k i} \}$ is a convergent sequence of $\FF$ for every $i \in 1,\dots,n$.
1. Thus, there exists $u_i = \lim_{n \to \infty} v_{k i}$
for every $i \in 1,\dots,n$.
1. Now, let $\bu = (u_1, \dots, u_n)$.
1. Then,
$$
\| \bv_k - \bu \| &\leq c_2 \| \bv_k - \bu \|_1 \\
&= c_2 \sum_{i=1}^n | v_{k i} - u_i |.
$$
1. Thus,
$$
\lim_{k \to \infty} \| \bv_k - \bu \|
&= \lim_{k \to \infty} c_2 \sum_{i=1}^n | v_{k i} - u_i | \\
&= c_2 \sum_{i=1}^n \lim_{k \to \infty} | v_{k i} - u_i | = 0.
$$
1. Thus, $\lim_{k \to \infty} \bv_k = \bu$.
1. Thus, $\{ \bv_k \}$ is convergent.
1. Thus, every Cauchy sequence of $\VV$ is convergent.
1. Thus, $\VV$ is complete.
```
## Compact Sets
In this subsection, we pay special attention to the
compact subsets of $n$-dimensional real normed linear
spaces. We show that in such spaces, closed and bounded
sets are compact.
We provides some results related to set arithmetic for
compact sets.
We further establish the
Bolzano Weierstrass theorems for such spaces.
Recall from {prf:ref}`def-ms-bolzano-weierstrass-property`
that a set has Bolzano-Weierstrass property
if every sequence of the set has a convergent subsequence
that converges to a point in the set.
```{prf:theorem} Compact = Closed and Bounded in $n$-dim
:label: res-la-ndim-compact-closed-bounded
Let $\VV$ be a real $n$-dimensional normed linear space.
Let $S$ be a subset of $\VV$.
Then, $S$ is compact if and only if $S$ is closed and bounded.
```
```{prf:proof}
By {prf:ref}`res-ms-compact-is-closed-bounded`,
every compact set is closed and bounded.
Thus, if $S$ is compact then $S$ is closed and bounded.
For the converse, assume that $S$ is closed and bounded.
1. By {prf:ref}`res-la-ndim-complete`, both $\RR^n$ and $\VV$
are complete.
1. Let $T: \RR^n \to \VV$ be an isomorphism.
1. Then, for any $\bv \in \RR^n$,
a function $\| \cdot \|_T : \RR^n \to \RR$ given by
$$
\| \bv \|_T = \| T (\bv)\|
$$
defines a norm on $\RR^n$.
1. The norm $\| \cdot \|_T$ is equivalent to the
Euclidean norm on $\RR^n$
due to {prf:ref}`res-la-ns-finite-all-norms-eq`.
1. Also, $T : (\RR^n, \| \cdot \|_T) \to (\VV, \| \cdot \|)$
is an {prf:ref}`isometry `
1. Accordingly $T$ is uniformly continuous by
{prf:ref}`res-ms-isometry-continuous`.
1. Since $T$ is an isomorphism, it is bijective.
1. Thus, $T$ is a homeomorphism by
{prf:ref}`res-ms-isometric-homeomorphic`.
1. Then, $T^{-1}$ is also an isometry.
1. Then, $T^{-1}(S)$ is bounded in $(\RR^n, \| \cdot \|_T)$
since $S$ is bounded in $(\VV, \| \cdot \|)$
and $T^{-1}$ is an isometry.
1. By {prf:ref}`res-la-ns-norm-eq-same-bounded`,
$T^{-1}(S)$ is also bounded in $(\RR^n, \| \cdot \|_2)$.
1. Also, $T^{-1}(S)$ is closed in $(\RR^n, \| \cdot \|_T)$.
since $S$ is closed in $(\VV, \| \cdot \|)$ and $T$ is continuous
(see {prf:ref}`res-ms-continuous-function-characterization`).
1. Since equivalent norms (metrics) determine same topology,
hence $T^{-1}(S)$ is closed in $(\RR^n, \| \cdot \|_2)$ also.
1. Thus, $T^{-1}(S)$ is closed and bounded in $(\RR^n, \| \cdot \|_2)$.
1. By {prf:ref}`Heine-Borel theorem `,
$T^{-1}(S)$ is a compact set in $(\RR^n, \| \cdot \|_2)$.
1. By {prf:ref}`res-ms-eq-metric-compactness`,
$T^{-1}(S)$ is a compact set in $(\RR^n, \| \cdot \|_T)$
also.
1. Then, $S = T (T^{-1}(S))$ is also compact
in $(\VV, \| \cdot \|)$
since a homeomorphism preserves compactness
(see {prf:ref}`res-ms-compact-homeomorphism-pres`).
```
```{prf:theorem} Sum of compact sets is compact
:label: res-la-ndim-sum-compact
Let $\VV$ be a real $n$-dimensional normed linear space.
Let $A, B \subseteq \VV$ be compact subsets of $\VV$.
Then, their sum $A+B$ is compact.
```
```{prf:proof}
We proceed as follows
1. Both $A$ and $B$ are compact. Hence, they are closed and bounded.
1. By {prf:ref}`res-la-sum-closed-compact`, $A+B$ is closed.
1. Let $\| \ba \| \leq M_a$ for every $\ba \in A$.
1. Let $\| \bb \| \leq M_b$ for every $\bb \in B$.
1. Let $\bx \in A + B$.
1. Then, there exists $\ba \in A$ and $\bb \in B$ such that
$\bx = \ba + \bb$.
1. Then,
$$
\| \bx \| = \| \ba + \bb \| \leq \| \ba \| + \| \bb \| \leq M_a + M_b.
$$
1. Thus, $\| \bx \| \leq M_a + M_b$ for every $\bx \in A + B$.
1. Thus, $A+B$ is bounded.
1. Since $A+B$ is closed and bounded, hence $A+B$ is compact
due to {prf:ref}`res-la-ndim-compact-closed-bounded`.
```
```{prf:theorem} Bolzano Weierstrass theorem for bounded subsets
:label: res-la-ndim-bounded-bolzano-weierstrass
Let $\VV$ be a real $n$-dimensional normed linear space.
Let $A$ be a bounded subset of $\VV$. Then,
every sequence of $A$ has a convergent subsequence.
If $A$ is closed, then the subsequence converges in $A$ itself.
Otherwise, the subsequence converges in $\closure A$.
```
```{prf:proof}
We are given that $A$ is bounded.
1. Then, there exist $M > 0$ such that:
$$
\| \bx \| \leq M \Forall \bx \in A.
$$
1. Thus, $A \subseteq B[\bzero, M]$.
1. $B[\bzero, M]$ is a closed and bounded subset of $\VV$.
1. $\closure A \subseteq B[\bzero, M]$ since $\closure A$ is the
smallest closed set containing $A$.
1. Thus, $\closure A$ is closed and bounded.
1. By {prf:ref}`res-la-ndim-compact-closed-bounded`,
$\closure A$ is compact.
1. Let $\{ \bx_k \}$ be a sequence of $A$. Then, it is also a sequence of $\closure A$.
1. By {prf:ref}`def-ms-compact-characterization`, $\{ \bx_k \}$ has
a subsequence that converges in $\closure A$.
1. If $A$ is closed, then $\closure A = A$ and we are done.
```
```{prf:theorem} Bolzano Weierstrass theorem for bounded sequences
:label: res-la-bounded-seq-bolzano-weierstrass
Let $\VV$ be a real $n$-dimensional normed linear space.
Every bounded sequence of $\VV$ has a convergent subsequence.
```
```{prf:proof}
Let $\{ \bx_k \}$ be a bounded sequence of $\VV$.
1. Then there exists a closed ball $B[\bzero, M]$ such that
$\{ \bx_k \} \subset B[\bzero, M]$.
1. $B[\bzero, M]$ is closed and bounded.
1. By {prf:ref}`res-la-ndim-compact-closed-bounded`,
$B[\bzero, M]$ is compact.
1. By {prf:ref}`def-ms-compact-characterization`, $\{ \bx_k \}$ has
a subsequence that converges in $B[\bzero, M]$.
```