3.8. Compactness#

The material in this section is primarily based on [2, 41].

Recall from Definition 1.21 that for a subset \(A \subseteq X\), a cover is a family \(\{ A_i \}_{i \in I}\) of subsets of \(X\) such that

\[ A \subseteq \bigcup_{i \in I} A_i. \]

3.8.1. Open Covers#

Definition 3.54 (Open cover)

A family of open subsets \(\{ A_i \}_{i \in I}\) of subsets of \((X,d)\) is an open cover of \(A\) if it covers \(A\).

Theorem 3.74 (Lindelöf)

Every open cover of a subset of \(\RR^m\) can be reduced to an at-most countable subcover.

Proof. We call a point \(a = (a_1, \dots, a_m) \in \RR^m\) a rational point if every component of \(a\) is a rational number.

Let \(A\) be a subset of \(\RR^m\).
Let \(\{ \OOO_i \}_{i \in I}\) be an open cover of \(A\) (possibly uncountable).
Thus, \(A \subseteq \bigcup_{i \in I} \OOO_i\).
For each \(x \in A\),
1. Choose an index \(i_x \in I\) such that \(x \in \OOO_{i_x}\).
2. Pick a rational point \(a_x \in \RR^m\) and a rational positive number \(r_x\) such that \(x \in B(a_x, r_x) \subseteq \OOO_{i_x}\).
Consider the collection \(C = \{ B(a_x, r_x) \ST x \in A \}\).
Since the set of rational points is countable and the set of rational numbers is countable, hence the set of all open balls centered at rational points with rational radii is countable.
Hence, \(C\) which is a subset of open balls with rational points as centers and rational radii, is at most countable.
Thus, \(C\) is an at most countable open cover of \(A\).
Since each \(B(a_x, r_x)\) is a subset of an \(\OOO_{i_x}\), hence there exists an at most countable subcover of \(\{ \OOO_i\}_{i \in I}\) for \(A\).

3.8.2. Compact Sets#

Definition 3.55 (Compact set)

Let \((X, d)\) be a metric space. A subset \(A\) of \(X\) is called compact if every open cover of \(A\) can be reduced to a finite subcover.

Definition 3.56 (Compact metric space)

Let \((X,d)\) be a metric space. If \(X\) is itself a compact set, then \((X,d)\) is called a compact metric space.

Example 3.24 (\((0,1)\) is not compact)

The set \((0,1)\) is not compact in \(\RR\). We first show this the hard way by picking an open cover for \((0,1)\) which cannot be reduced to a finite subcover. Later, we discuss how to verify compactness through easy checks.

Consider the family of open intervals:

\[ C = \left \{ \left(\frac{1}{n}, 1 \right ) \ST n \geq 2 \right \}. \]
For every \(x \in (0,1)\), there is a natural number \(n\) such that \(x > \frac{1}{n}\).
Thus, \(x \in \left(\frac{1}{n}, 1 \right )\).
Thus,

\[ (0, 1) \subseteq \bigcup_{n=2}^{\infty} \left(\frac{1}{n}, 1 \right ) \]

implying that \(C\) is an open cover of \((0,1)\).
At the same time for every \(n \geq 2\), we have:

\[ \left(\frac{1}{n}, 1 \right ) \subseteq (0, 1) \]

since \(\frac{1}{n} > 0\).
Thus,

\[ (0, 1) = \bigcup_{n=2}^{\infty} \left(\frac{1}{n}, 1 \right ). \]
But there is no finite subcover of \((0,1)\) in \(C\).
If there was a finite subcover, we could pick a maximum \(n\) among those intervals.
But then \(x = \frac{1}{n}\) won’t belong to any of those intervals in the finite subcover.
Hence \((0,1)\) is not compact.

We later show in Theorem 3.76 that every compact set is closed and bounded. Hence, an easy way to say that \((0,1)\) is not compact is by noticing that it is not closed.

3.8.3. Characterization of Compactness#

We have defined compactness as a property where every open cover can be reduced to a finite subcover. The characterization of a property involves identifying other properties which are equivalent in the sense that property A \(\iff\) property B. If one is true then the other must be true and vice versa.

We will see later that compactness of a set \(A\) is equivalent to the property that every sequence of \(A\) has a subsequence that converges to a point in \(A\). However, before we go there, let us examine some implications of this property that every sequence of a set \(A\) has a subsequence that converges within the set \(A\). These results will be useful later in the characterization of compact sets.

Lemma 3.1 (Lebesgue number)

Let \(A \subseteq \bigcup_{i \in I} \OOO_i\) be an open cover of \(A\).

If every sequence in \(A\) has a subsequence which converges to a point of \(A\), then, there exists a number \(\delta > 0\) such that for each \(x \in A\), we have \(B(x, \delta) \subseteq \OOO_i\) for at least one \(i \in I\).

Any such number \(\delta > 0\) is called a Lebesgue number of \(A\) for the open cover \(\{\OOO_i \}_{i \in I}\).

Proof. Assume the claim is false.

Then, for each \(\delta > 0\), there exists some \(x \in A\) such that \(B (x, \delta)\) is not a subset of any \(\OOO_i\).
In particular, for each \(n\), there exists some \(x_n \in A\) such that \(B (x_n, \frac{1}{n}) \cap (X \setminus \OOO_i) \neq \EmptySet\) holds for each \(i \in I\).
Consider the sequence \(\{ x_n \}\).
By the hypothesis, every sequence has a subsequence that converges to a point of \(A\).
Let \(x \in A\) be the limit of such a subsequence of \(\{ x_n \}\).
Since \(x \in A\), \(x \in \OOO_i\) for at least one \(i \in I\).
Pick some \(i \in I\) such that \(x \in \OOO_i\).
Choose some \(r > 0\) such that \(x \in B(x, r) \subseteq \OOO_i\).
1. Recall that \(\OOO_i\) is open and \(x\) is its interior point. So such an \(r > 0\) can be chosen.
Now, select some \(n\) such that \(\frac{1}{n} < \frac{r}{2}\) and \(d(x, x_n) < \frac{r}{2}\).
It follows that \(B(x_n, \frac{1}{n}) \subseteq B(x, r) \subseteq \OOO_i\).
But this contradicts with the selection of \(x_n\) such that \(B (x_n, \frac{1}{n})\) is not contained in any \(\OOO_i\).
Thus, a \(\delta > 0\) must exist satisfying the condition that for each \(x \in A\), \(B(x, \delta) \subseteq \OOO_i\) for at least one \(i \in I\).

Lemma 3.2 (Existence of finite cover of open balls)

Let \(A \subseteq X\) of a metric space \((X,d)\). If every sequence in \(A\) has a subsequence which converges to a point of \(A\), then, for every \(r > 0\) there exist \(x_1, \dots, x_n \in A\) such that

\[ A \subseteq \bigcup_{j=1}^n B(x_j, r). \]

This lemma simply claims that we can construct a finite open cover of open balls for \(A\) for every \(r > 0\).

Proof. Assume the claim to be false. Choose \(r > 0\) such that it is not possible to select a finite number of points from \(A\) such that

\[ A \subseteq \bigcup_{j=1}^n B(x_j, r). \]

Pick some \(x_1 \in A\).
Choose some \(x_2 \in A \setminus B(x_1, r)\). We can choose \(x_2\) since \(A\) doesn’t have a finite open ball cover.
Assuming that \(x_1, x_2, \dots, x_n\) have been chosen inductively, choose \(x_{n+1}\) from the set \(A \setminus \bigcup_{i=1}^n B(x_i, r)\).
By design, \(d(x_n, x_m) \geq r\) holds for every \(n \neq m\).
Then, no subsequence of \(\{ x_n \}\) can converge.
This contradicts with the hypothesis that every sequence has a subsequence that converges in \(A\).
Hence, the claim must be true.

Theorem 3.75 (Characterization of compactness)

Let \(A\) be a subset of a metric space \((X,d)\). The following statements are equivalent.

\(A\) is compact.
Every infinite subset of \(A\) has an accumulation point in \(A\).
Every sequence in \(A\) has a subsequence which converges to a point of \(A\).

Proof. (1) \(\implies\) (2)

Let \(S\) be an infinite subset of \(A\).
Assume, by way of contradiction, that \(S\) has no accumulation point in \(A\).
Then for every \(x \in A\), there exists a deleted neighborhood of \(x\) that is disjoint with \(S\).
That is, \(\forall x \in A, \exists r_x > 0 \ST B (x, r_x) \cap ( S \setminus \{ x \}) = \EmptySet\).
Then, \(B(x, r_x) \cap S\) can either be empty or it can at most contain \(x\).
Thus, \(B(x, r_x) \cap S \subseteq \{ x\}\).
Consider the open cover \(C = \bigcup_{x \in A} B(x, r_x)\).
It is easy to see that \(A \subseteq C\).
Since \(A\) is compact (by hypothesis), there exists a finite set \(\{ x_1, \dots, x_n \} \subseteq A\) such that \(A \subseteq \bigcup_{i=1}^n B(x_i, r_{x_i})\).
But then

\[ S = A \cap S = \bigcup_{i=1}^n [B(x_i, r_{x_i}) \cap S] \subseteq \{ x_1, \dots, x_n \}. \]
Thus, \(S\) must be a finite set containing at most \(n\) elements.
We arrive at a contradiction as \(S\) is infinite.
Hence, \(S\) has an accumulation point in \(A\).

(2) \(\implies\) (3)

We assume that every infinite subset of \(A\) has an accumulation point in \(A\).
Let \(\{x_n \}\) be an arbitrary sequence of \(A\).
If \(\{x_n \}\) is constant, then every subsequence is constant and convergent.
If \(\{x_n \}\) takes a finite number of distinct values, then there is at least one value which must occur infinite times. Thus, at least one subsequence is a constant sequence and hence convergent.
We are left with the case where \(\{ x_n \}\) contains infinite distinct values.
Then, we can choose a subsequence \(\{ y_n \}\) of \(\{ x_n \}\) which consists of all distinct values; i.e., \(y_n \neq y_m\) whenever \(n \neq m\).
1. Let \(k_1 = 1\).
2. Assuming \(k_1, \dots, k_n\) have been selected, choose \(k_{n+1} > k_n\) such that \(x_{k_{n+1}} \neq x_{k_i}\) for \(1 \leq i \leq n\).
3. This is possible since \(\{x_n \}\) has infinite distinct values and so far only \(n\) distinct values have been chosen.
4. Now, let \(y_n = x_{k_n}\).
5. It is clear that \(\{y_n\}\) is a subsequence of \(\{x_n\}\) consisting of all distinct values.
Consider the set \(Y = \{ y_1, y_2, \dots \}\) (not the sequence but the set).
By our hypothesis (2), since \(Y\) is an infinite subset of \(A\), hence it must have an accumulation point in \(A\).
Let \(x\) be an accumulation point of \(Y\).
Assume that \(x \neq y_n\) for each \(n\). If \(x = y_k\) for some \(k\), then drop the first \(k\) elements of \(\{ y_n \}\).
This ensures that \(d(x, y_n) > 0\) for every \(n\).
We will now select a subsequence of \(\{ y_n \}\) that converges to \(x\).
Towards this, choose \(m_1\) such that \(d(y_{m_1}, x) < 1\).
Now, inductively, assuming that \(m_1 < m_2, \dots < m_n\) have been chosen, choose \(m_{n+1}\) such that

\[ d(y_{m_{n+1}}, x) < r_{n+1} = \min \{\frac{1}{n+1}, d(y_1, x), d(y_2, x), \dots, d(y_{m_n}, x) \}. \]
Since \(d(x, y_n) > 0\), hence \(r_{n+1} > 0\).
Since \(x\) is an accumulation point of \(Y\), hence for every \(r > 0\), there exists a point \(y \in Y\) such that \(d(x, y) < r\).
Thus, it is possible to pick a suitable \(m_{n+1}\) such that \(d(y_{m_{n+1}}, x) < r_{n+1}\).
Also, by design, \(m_{n+1}\) must be greater than \(m_n\) as \(r_{n+1} < d(y_i, x) \Forall 1 \leq i \leq m_n\).
Thus, \(\{ y_{m_n} \}\) is a subsequence of \(\{ y_n \}\).
Hence, \(\{y_{m_n}\}\) is a subsequence of \(\{ x_n \}\) too.
Since \(d(x, y_{m_n}) < \frac{1}{n}\), it follows that \(\lim y_{m_n} = x\).
Thus, \(\{ x_n \}\) has a convergent subsequence which converges to a point of \(A\).

(3) \(\implies\) (1)

Let \(A \subseteq \bigcup_{i \in I} \OOO_i\) be an arbitrary open cover of \(A\).
We can pick a Lebesgue number \(\delta > 0\) such that for each \(x \in A\), we have \(B(x, \delta) \subseteq \OOO_i\) for at least one \(i \in I\) thanks to Lemma 3.1.
Now, thanks to Lemma 3.2, we can pick \(x_1, \dots, x_n \in A\) such that:

\[ A \subseteq \bigcup_{j=1}^n B(x_j, \delta). \]
Now, for each \(j\) pick some \(i_j \in I\) such that \(B(x_j, \delta) \subseteq \OOO_{i_j}\).
Then,

\[ A \subseteq \bigcup_{j=1}^n B(x_j, \delta) \subseteq \bigcup_{j=1}^n \OOO_{i_j}. \]
Thus, the open cover of \(A\) has a finite subcover.
Thus, \(A\) is compact.

Definition 3.57 (Bolzano-Weierstrass property)

A set \(A\) in a metric space has the Bolzano-Weierstrass property if every sequence in \(A\) has a convergent subsequence that converges to a point in \(A\).

Observation 3.1

A compact set has the Bolzano-Weierstrass property.

3.8.4. Closedness and Boundedness#

Theorem 3.76 (Compact sets are closed and bounded)

A compact set is closed and bounded.

Proof. Assume \(A\) to be compact. We shall show that \(A\) must be bounded.

Choose an open cover for \(A\) as \(A \subseteq \bigcup_{x \in A}B(x, 1)\).
Since \(A\) is compact, hence there exist finite set of points \(\{ x_1, \dots x_n \} \subseteq A\) such that \(A \subseteq \bigcup_{i=1}^n B(x_i, 1)\).
Let \(M = \max \{ d(x_i, x_j) \ST 1 \leq i, j \leq n\}\).
For any \(x, y \in A\), choose \(i, j\) such that \(x \in B(x_i, 1)\) and \(y \in B(x_j, 1)\).
Then, by triangle inequality, we have:

\[ d(x, y) \leq d(x, x_i) + d(x_i, x_j) + d(x_j, y) < M + 2 < \infty. \]
Thus, \(\diam A = \sup d(x,y)\) is finite and \(A\) is bounded.

We now show that if \(A\) is compact then \(A\) must be closed too. Towards this, we show that \(A\) contains all its closure points.

Let \(x \in \closure A\).
Due to Theorem 3.31, there exists a sequence \(\{ x_n \}\) of \(A\) with \(\lim x_n = x\).
Due to Theorem 3.75, \(\{x_n\}\) has a subsequence that converges to a point of \(A\) (since \(A\) is compact by hypothesis).
As per Theorem 3.35, subsequences of a convergent sequence converge to the same limit.
Thus, \(x\) must be in \(A\).
Thus, \(\closure A \subseteq A\). Thus, \(\closure A = A\). Thus, \(A\) is closed.

Although every compact set is closed and bounded, the converse need not be true. See Example 3.29 for an example of closed and bounded set (in discrete space) which is not compact. In fact, discrete space is a complete metric space. Yet, it has closed and bounded sets which are not compact.

In the specific case of Euclidean spaces, all closed and bounded sets are compact too. See Heine-Borel theorem below.

3.8.5. Nested Sequence of Compact Sets#

Theorem 3.77 (Nonempty nested intersection property)

Let \(\{ A_n \}\) be a nested sequence of nonempty compact subsets of \((X, d)\) such that

\[ S_1 \supseteq S_2 \supseteq S_3 \supseteq \dots \]

Then their intersection is nonempty; i.e.,

\[ \bigcap_{n=1}^{\infty} S_n \neq \EmptySet. \]

Proof. We prove this result by way of contradiction.

Assume for contradiction that \(\bigcap_{n=1}^{\infty} S_n = \EmptySet\).
Let \(V_n = X \setminus S_n\) for every \(n \in \Nat\).
Since \(S_{n +1 } \subseteq S_n\) for every \(n\) hence \(V_n \subseteq V_{n+1}\) for every \(n\).
Then \(V = \{ V_n \ST n \in \Nat \}\) is a collection of open sets.
Since \(\bigcap_{n=1}^{\infty} S_n = \EmptySet\), hence

\[\begin{split} X &= X \setminus \EmptySet \\ &= X \setminus \left ( \bigcap_{n=1}^{\infty} S_n \right ) \\ &= \bigcup_{n=1}^{\infty}(X \setminus S_n) \\ &= \bigcup_{n=1}^{\infty} V_n. \end{split}\]
Hence \(V\) is an open cover for every \(S_n\) as \(S_n \subseteq X = \bigcup_{n=1}^{\infty} V_n\).
In particular, \(V\) is an open cover for \(S_1\).
Since \(S_1\) is compact, hence there exists a finite subcover of \(S_1\) indexed by a finite set of natural numbers \(I\).
Let \(I = \{ i_1, i_2, \dots, i_k \}\) and the finite subcover be \(F_k = \{ V_i \in V \ST i \in I \}\).
Since \(I\) is a finite set, it has a maximum element.
Let \(m = \max I = \max \{ i_1, i_2, \dots, i_k \}\).
Then \(F_m = \{ V_1, \dots, V_m \}\) is also a finite subcover of \(S_1\) since \(F_k \subseteq F_m\).
But then \(\bigcup_{k=1}^m V_k = V_m\) since \(V_n \subseteq V_{n+1}\) for every \(n\).
Hence \(S_1 \subseteq V_m\).
Then \(S_m \subseteq S_1 \subseteq V_m\).
But \(V_m = X \setminus S_m\), hence \(V_m \cap S_m = \EmptySet\) a contradiction.
Hence, the intersection of the nested compact sets must be nonempty.

3.8.6. Continuity#

Theorem 3.78 (Continuous images of compact sets are compact)

Let \(f: (X, d) \to (Y, \rho)\) be a continuous function. Let \(A\) be a compact subset of \(X\) with \(A \subseteq \dom f\). Then \(f(A)\) is a compact subset of \(Y\).

Proof. We prove this by showing that any open cover of \(f(A)\) can be reduced to a finite subcover.

Let \(f(A) \subseteq \bigcup_{i \in I} \OOO_i\) be an open cover for \(f(A)\).
Then \(A \subset \bigcup_{i \in I} f^{-1} (\OOO_i)\).
Since \(f\) is continuous, hence \(f^{-1} (\OOO_i)\) is an open subset of \(X\) for every \(i \in I\).
Since \(A\) is compact, there exist indices \(i_1, \dots, i_n\) such that \(A \subseteq \bigcup_{j=1}^n f^{-1}(\OOO_{i_j})\).
Then

\[ f(A) \subseteq f\left ( \bigcup_{j=1}^n f^{-1}(\OOO_{i_j}) \right ) = \bigcup_{j=1}^n f(f^{-1}(\OOO_{i_j})) \subseteq \bigcup_{j=1}^n \OOO_{i_j}. \]
Thus, \(A\) is compact.

3.8.7. Lipschitz Continuity#

Theorem 3.79

Let \(f : (X,d) \to (Y, \rho)\) be a (partial) function with \(S = \dom f\). Assume that \(f\) is locally Lipschitz continuous at every \(x \in S\). Let \(A \subseteq S\) be a compact subset of \(S\). Then, \(f\) is Lipschitz function on \(A\). In other words, there exists a constant \(L > 0\) such that

\[ \rho(f(x),f(y)) \leq L d(x, y) \]

for every \(x, y \in A\).

Proof. We proceed as follows.

Since \(f\) is locally Lipschitz continuous on \(S\) hence \(f\) is continuous by Theorem 3.57.
Thus, by Theorem 3.78, \(f(A)\) is compact.
Hence, \(f(A)\) is closed and bounded.
Thus, there exists \(M > 0\) such that for any \(x, y \in A\), \(\rho(f(x), f(y)) \leq M\).
For contradiction, assume that \(f\) is not Lipschitz on \(A\).
Then, there is no \(L > 0\) such that

\[ \rho(f(x), f(y)) \leq L d(x,y) \Forall x, y \in A. \]
Then, there exist two sequences \(\{ x_n \}\) and \(\{ y_n \}\) of \(A\) such that

\[ \lim_{n \to \infty} \frac{\rho(f(x_n), f(y_n)) }{d(x_n,y_n)} = \infty. \]
But \(\rho(f(x_n), f(y_n)) \leq M\).
Thus,

\[ \lim d(x_n,y_n) = 0. \]
Since \(A\) is compact, hence \(\{ x_n \}\) has a convergent subsequence.
Let \(\{x_{n_k} \}\) be a convergent subsequence with \(x = \lim_{k \to \infty} x_{n_k}\).
By compactness of \(A\), \(x \in A\).
Then, \(f\) cannot be not locally Lipschitz continuous at \(x\).
This contradicts our hypothesis.
Thus, \(f\) must be Lipschitz continuous on \(A\).

3.8.8. Homeomorphism#

Theorem 3.80 (Homeomorphism preserves compactness)

Let \(f: (X, d) \to (Y, \rho)\) be a homeomorphism. Then, \(A\) is a compact subset of \((X,d)\) is and only if \(f(A)\) is a compact subset of \((Y, \rho)\).

Proof. Let \(A \subseteq X\) be compact. Then, \(f(A)\) is compact since \(f\) is continuous due to Theorem 3.78.

Let \(f(A)\) be compact. Since \(f\) is a homeomorphism, hence \(f^{-1}\) is continuous and bijective. Hence, \(f^{-1}(f(A)) = A\) is compact due to Theorem 3.78.

3.8.9. Compact Spaces#

Theorem 3.81

Every closed subset of a compact space is compact.

Proof. Let \((X, d)\) be a compact metric space and let \(A\) be a closed subset of \(X\).

Let \(\{\OOO_i\}_{i \in I}\) be an open cover of \(A\). We have, \(A \subseteq \bigcup_{i \in I}\OOO_i\).
\(X = A \cup (X \setminus A)\).
Then, \(X \subseteq (X \setminus A) \cup \bigcup_{i \in I} \OOO_i\).
Since all \(\OOO_i\) are subsets of \(X\), hence we can write it as: \(X = (X \setminus A) \cup \bigcup_{i \in I} \OOO_i\).
Since \(A\) is closed, hence \(X \setminus A\) is open.
Thus, \((X \setminus A) \cup \bigcup_{i \in I} \OOO_i\) is an open cover of \(X\).
But \(X\) is compact. Hence, there exist finite indices \(i_1, \dots, i_n\) such that \(X = (X \setminus A) \cup \OOO_{i_1} \cup \dots \cup \OOO_{i_n}\).
But then \(A \subseteq X\) and \(A \cap (X \setminus A) = \EmptySet\) imply that: \(A \subseteq \OOO_{i_1} \cup \dots \cup \OOO_{i_n}\).
Thus, \(A\) is compact.

Theorem 3.82 (Continuous maps are closed)

Let \((X, d)\) be a compact metric space and suppose that \(f : (X, d) \to (Y, \rho)\) is a (total) continuous function. Then \(f\) is a closed mapping. If \(f\) is bijective, then \(f\) is a homeomorphism.

Proof. Let \(C\) be a closed subset of \(X\).

Due to Theorem 3.81, \(C\) is compact.
Since \(f\) is continuous, hence, due to Theorem 3.78, \(f(A)\) is compact.
As per Theorem 3.76, since \(f(A)\) is compact, hence \(f(A)\) is closed.
Thus, \(f\) maps every closed set to a closed set.
Thus, \(f\) is a closed mapping.

Now, assume that \(f\) is bijective too.

Thus, \(f^{-1}\) exists. Let \(g = f^{-1}\).
Then, due to bijection property \(g^{-1}(A) = f(A)\) holds for every subset \(A\) of \(X\).
Thus, \(g^{-1}(A) = f(A)\) is a closed subset of \(Y\) whenever \(A\) is a closed subset of \(X\).
Thus, as per Theorem 3.42 \([(5) \implies (1)]\), \(g\) is continuous.
Thus, both \(f\) and \(f^{-1} = g\) are continuous.
Thus, \(f\) is a homeomorphism.

Theorem 3.83 (Compact domain + Continuity = Uniform continuity)

Let \(f: (X, d) \to (Y, \rho)\) be continuous on \(X\). If \(X\) is compact, then \(f\) is uniformly continuous.

Proof. We proceed as follows.

Let \(\epsilon > 0\).
Since \(f\) is continuous on \(X\), hence for every \(x \in X\), there exists \(r_x > 0\) such that \(\rho(f(y), f(x)) < \epsilon\) holds whenever \(d(x, y) < 2 r_x\).
The collection of open balls \(B(x, r_x)\) covers \(X\); i.e., \(X = \bigcup_{x \in X}B(x, r_x)\).
Since \(X\) is compact, there exists a set of finite number of points \(x_1, \dots, x_n\) such that \(X = \bigcup_{i=1}^n B(x_i, r_{x_i})\).
Now, let \(\delta = \min \{r_{x_1}, \dots, r_{x_n} \}\).
Since \(\delta\) is the minimum of a finite number of positive numbers, hence \(\delta > 0\).
Now, pick any \(x, y \in X\) that satisfy \(d(x, y) < \delta\).
There exists an integer \(i\) such that \(d(x, x_i) < r_{x_i}\) (due to the finite open cover).
Therefore, \(\rho(f(x), f(x_i)) < \epsilon\).
Now, by triangle inequality:

\[ d(y, x_i) \leq d(y, x) + d(x, x_i) < \delta + r_{x_i} \leq 2 r_{x_i} \]

holds true.
Thus, \(\rho(f(x_i), f(y)) < \epsilon\), since \(d(y, x_i) < 2 r_{x_i}\).
Thus,

\[ \rho(f(x), f(y)) \leq \rho(f(x), f(x_i)) + \rho(f(x_i), f(y)) < \epsilon + \epsilon = 2\epsilon. \]
Thus, \(f\) is uniformly continuous.

3.8.10. Euclidean Spaces#

Recall that \(\RR^m\) are called Euclidean spaces with the standard metric:

\[ d (x, y) \triangleq \left ( \sum_{i=1}^m |x_i - y_i|^2 \right )^{\frac{1}{2}}. \]

By \(0 \in \RR^m\) we shall mean the vector \((0, \dots, 0)\).

The compact subsets of a Euclidean space are precisely those sets which are closed and bounded.

Theorem 3.84 (Heine-Borel theorem)

A subset of a Euclidean space is compact if and only if it is closed and bounded.

Compare this to Heine-Borel theorem for the real line. We established there that for a closed and bounded subset of \(\RR\), any open cover can be reduced to a finite subcover. There, we defined closed and bounded sets as compact sets. Our treatment of compactness in this section is more general.

We start here with the definition that a compact set is one for which any open cover can be reduced to a finite subcover. We then show in this theorem that in the special case of Euclidean spaces \(\RR^m\), the compact subsets are identical to the closed and bounded subsets of \(\RR^m\).

Proof. We have shown in Theorem 3.76 that every compact set is closed and bounded.

For the converse, we assume \(A\) is a closed and bounded subset of \(\RR^m\). We will show that every sequence of \(A\) has a subsequence converging in \(A\).

Since \(A\) is bounded, we can pick \(M > 0\) such that \(d(x, y) \leq M\) for all \(x, y \in A\).
Fix an element \(y \in A\).
Let \(a = (a_1, \dots, a_m) \in A\) be some arbitrary point of \(A\). Then

\[ |a_i | \leq d(a, 0) \leq d(a, y) + d(y, 0) \leq M + d(y, 0) \]

holds for every \(1 \leq i \leq m\).
Thus, the set of real numbers consisting of the \(i\)-th coordinates of the elements of \(A\) is a bounded set.
Choose an arbitrary sequence \(\{ x_n \}\) of \(A\).
Recall from Bolzano Weierstrass theorem for real numbers that every bounded sequence of real numbers has a convergent subsequence.
Note that if we form the sequence of real numbers from any particular coordinate of \(\{x_n\}\), (say first coordinates or second coordinates) then the sequence is bounded by \(M + d(y, 0)\).
Every such sequence of real numbers (from a fixed coordinate of \(\{x_n\}\)) will have a convergent subsequence.
Thus, there is a subsequence \(\{x^1_n\}\) of \(\{x_n\}\) whose first coordinates form a sequence in \(\RR\) that converges in \(\RR\).
Now, we choose \(\{x^2_n\}\) as a subsequence of \(\{x^1_n\}\) so that the corresponding sequence of second coordinates of \(\{x^2_n\}\) converge in \(\RR\).
Proceeding in this manner, after \(m\) steps, we have a subsequence \(\{x^m_n\}\) of \(\{x_n\}\) with the property that for each \(1 \leq i \leq m\), the sequence of its \(i\)-th coordinates forms a convergent subsequence in \(\RR\).
Since each of the coordinates of \(\{x^m_n\}\) converges in \(\RR\), hence, \(\{x^m_n\}\) converges in \(\RR^m\).
But since \(A\) is closed, hence \(\{x^m_n\}\) converges to a point of \(A\).
Thus, every sequence of \(A\) has a convergent subsequence in \(A\).
Thus, by Theorem 3.75, \(A\) is compact.

Note that the property convergence in individual coordinates implies convergence in \(\RR^m\) is due to the specific choice of Euclidean metric.

Theorem 3.85 (Attainment of minimum and maximum values)

Let \(f : (X, d) \to \RR\) be a real valued function. If \(f\) is continuous then \(f\) attains a maximum and minimum value on every compact subset of \(\dom f\).

Proof. Let \(A\) be a compact subset of \(\dom f\).

Then, \(f(A)\) is compact in \(\RR\) due to Theorem 3.78.
Then, \(f(A)\) is closed and bounded due to Heine-Borel theorem.
Since \(f(A)\) is bounded, hence it has an infimum and supremum.
Since \(f(A)\) is closed, hence its infimum and supremum lie inside \(f(A)\) itself.
Thus, \(f\) attains a maximum and minimum value in \(A\).

Theorem 3.86 (Bolzano Weierstrass theorem for bounded subsets of \(\RR^m\))

Every sequence in a closed and bounded set in \(\RR^m\) has a convergent subsequence.

Proof. Let \(A\) be a closed and bounded subset of \(\RR^n\).

By Theorem 3.84, \(A\) is compact.
By Theorem 3.75, every sequence in \(A\) has a subsequence which converges to a point of \(A\).

Theorem 3.87 (Bolzano Weierstrass theorem for bounded sequences of \(\RR^m\))

Every bounded sequence in \(\RR^m\) has a convergent subsequence.

Proof. Let \(\{ x_m \}\) be a bounded sequence of \(\RR^m\). Then there exists a closed ball \(B[0, M]\) such that \(\{ x_m \} \subset B[0, M]\).

\(B[0, M]\) is closed and bounded. From Theorem 3.86, every sequence in a closed and bounded subset of \(\RR^n\) has a convergent subsequence.

3.8.11. Totally Bounded Metric Spaces#

Definition 3.58 (Totally bounded space)

A metric space \((X,d)\) is called totally bounded if for each \(r > 0\), there exists a finite number of points \(x_1, \dots, x_n \in X\) such that

\[ X = \bigcup_{i=1}^n B(x_i, r). \]

Theorem 3.88 (Compactness implies total boundedness)

A compact metric space is totally bounded.

Proof. Let \((X,d)\) be a compact metric space.

Let \(r > 0\).
Consider the family of open balls \(\{ B(x, r) \}_{x \in X}\).
Then \(X = \bigcup_{x \in X} B(x, r)\).
Since \(X\) is compact, there is a finite subcover of open balls.
Thus, for every \(r > 0\), there \(X\) is a union of finite open balls.
Thus, \(X\) is totally bounded.

Example 3.25

We showed earlier in Example 3.24 that \((0,1)\) is not compact.

However \((0,1)\) is totally bounded.

Let \(r > 0\).
Pick any \(n> \frac{1}{r}\). Thus, \(r > \frac{1}{n}\).
Consider the points \(\frac{1}{n}, \frac{2}{n}, \dots, \frac{n-1}{n}\).
\(\frac{k}{n} + r > \frac{k}{n} + \frac{1}{n} = \frac{k+1}{n}\).
\(\frac{k}{n} - r < \frac{k}{n} - \frac{1}{n} = \frac{k-1}{n}\).
Thus,

\[ \left (\frac{k-1}{n}, \frac{k+1}{n}\right ) \subseteq B\left (\frac{k}{n}, r \right ) \]
Thus,

\[ (0, 1) = \bigcup_{k=1}^{n-1}B\left (\frac{k}{n}, r \right ). \]

with the caveat that the first and last balls are restricted within the set \((0,1)\).
Thus, we have a finite union of open balls.
Thus, \((0,1)\) is totally bounded.

Theorem 3.89 (Completeness and Compactness)

A metric space is compact if and only if it is complete and totally bounded.

Proof. Let \((X,d)\) be a metric space. Assume that \((X,d)\) is compact.

\((X,d)\) is totally bounded (Theorem 3.88).
Since \((X,d)\) is compact, hence every sequence has a convergent subsequence that converges to a point in \(X\) (Theorem 3.75 (3)).
Thus, if \(\{x_n\}\) is a Cauchy subsequence of \(X\), then it has a subsequence with limit \(x \in X\) leading to \(\lim x_n = x\).
Thus, every Cauchy sequence of \(X\) converges in \(X\).
Thus, \((X,d)\) is complete.

For the converse, assume that \((X,d)\) is complete and totally bounded. To show that \((X,d)\) is compact, we will show that every infinite subset of \(X\) has an accumulation point in \(X\) (Theorem 3.75 (2)).

Let \(A\) be an infinite subset of \(X\).
Since \(X\) is totally bounded, there exists a finite subset \(F_1 \subset X\) such that

\[ X = \bigcup_{x \in F_1} B(x, 1). \]
We can extend the open balls with closed balls without problem:

\[ X = \bigcup_{x \in F_1} B[x, 1]. \]
Thus:

\[ A = A \cap X = \bigcup_{x \in F_1}( A \cap B[x, 1]). \]
Since \(A\) is infinite, there is some \(x_1 \in F_1\) such that \(A_1 = A \cap B[x_1, 1]\) is an infinite set.
1. If \(A \cap B[x, 1]\) were finite for each \(x \in F\), then \(A\) would be finite as a finite union of finite sets.
Since \(X\) is totally bounded, we can again find a finite subset \(F_2 \subset X\) such that

\[ X = \bigcup_{x \in F_2} B\left [x, \frac{1}{2}\right]. \]
Since \(A_1\) is infinite, there is some \(x_2 \in F_2\) such that \(A_2 = A_1 \cap B\left[x_2, \frac{1}{2} \right]\) is an infinite set.
Proceeding inductively, if \(x_1, x_2, \dots, x_n\) have been chosen, we can choose \(x_{n+1}\) such that the set

\[ A \cap B[x_1, 1] \cap B\left [x_2, \frac{1}{2} \right] \cap \dots \cap B\left [x_n, \frac{1}{n}\right] \cap B\left [x_{n+1}, \frac{1}{n+1} \right] \]

is infinite.
Define

\[ E_n = B[x_1, 1] \cap B\left [x_2, \frac{1}{2} \right] \cap \dots \cap B\left [x_n, \frac{1}{n}\right] \]

for each \(n\).
Then for each \(n\):
1. \(E_n\) is nonempty and closed.
2. \(A \cap E_n\) is infinite.
3. \(E_{n+1} \subseteq E_{n}\).
4. \(\diam E_n \leq \frac{2}{n}\). \(\lim \diam E_n = 0\).
Due to Theorem 3.59, there exists \(a \in X\), such that \(a \in E_n\) for each \(n\).
Now, if \(y \in A \cap E_n\), then

\[ d(a, y) \leq d(a, x_n) + d(x_n, y) < \frac{1}{n} + \frac{1}{n} = \frac{2}{n}. \]
Thus, for every \(r > 0\), we can pick \(n > \frac{2}{r}\) such that for every \(y \in A \cap E_n\), \(y \in B(a, r)\).
Thus, \(a\) is an accumulation point of \(A\).
Thus, every infinite subset of \(X\) has an accumulation point in \(X\).
Thus, \(X\) is compact.

3.8.12. Equivalent Metrics#

Theorem 3.90 (Metric equivalence and compactness)

Let \(d_a\) and \(d_b\) be two different metrics on the same set \(X\) that are equivalent.

Then, a set \(A \subseteq X\) is compact in \((X, d_a)\) if and only if \(A\) is compact in \((X, d_b)\).

In other words, the compact sets in the two metric spaces are identical.

Proof. Assume \(A\) to be compact in \((X, d_a)\).

Let \(A \subseteq \bigcup_{i \in I} \OOO_i\) be an open cover of \(A\) in \((X, d_b)\); i.e. \(\OOO_i\) are open in \((X, d_b)\).
Since \(d_a\) and \(d_b\) are equivalent, hence \(\OOO_i\) are open in \((X, d_a)\) too.
Thus, \(\bigcup_{i \in I} \OOO_i\) is an open cover for \(A\) in \((X, d_a)\) too.
Since \(A\) is compact in \((X, d_a)\), hence, there exist finite indices \(i_1, \dots, i_n\) such that \(A \subseteq \OOO_{i_1} \cup \dots \cup \OOO_{i_n}\).
But then, \(\OOO_{i_1}, \dots, \OOO_{i_n}\) are open in \((X, d_b)\) too.
Thus, \(A \subseteq \OOO_{i_1} \cup \dots \cup \OOO_{i_n}\) is a finite open subcover of \(A\) in \((X, d_b)\).
Thus, every open cover of \(A\) in \((X, d_b)\) can be reduced to a finite subcover.
Thus, \(A\) is compact in \((X, d_b)\).

A similar reasoning establishes that if \(A\) is compact in \((X, d_b)\) then \(A\) is compact in \((X, d_a)\) too.

Topics in Signal Processing

Compactness

Contents

3.8. Compactness#

3.8.1. Open Covers#

3.8.2. Compact Sets#

3.8.3. Characterization of Compactness#

3.8.4. Closedness and Boundedness#

3.8.5. Nested Sequence of Compact Sets#

3.8.6. Continuity#

3.8.7. Lipschitz Continuity#

3.8.8. Homeomorphism#

3.8.9. Compact Spaces#

3.8.10. Euclidean Spaces#

3.8.11. Totally Bounded Metric Spaces#

3.8.12. Equivalent Metrics#