4.8. Matrices III

4.8.1. Orthogonal Matrices

Definition 4.109

A real square matrix $\mathbf{U}$ is called orthogonal if the columns of $\mathbf{U}$ form an orthonormal set. In other words, let

$$\mathbf{U}=\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]$$

with ${\mathbf{u}}_i\in {\mathbb{R}}^n$. Then we have

$${\mathbf{u}}_i\cdot {\mathbf{u}}_j={\delta }_{i,j}.$$

Lemma 4.44

An orthogonal matrix $\mathbf{U}$ is invertible with ${\mathbf{U}}^T={\mathbf{U}}^{-1}$.

Proof. Let

$$\mathbf{U}=\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]$$

be orthogonal with

$$\begin{array}{r}{\mathbf{U}}^T=\left[\begin{array}{c}{\mathbf{u}}_1^T\\ {\mathbf{u}}_2^T\\ ⋮\\ {\mathbf{u}}_n^T\end{array}\right].\end{array}$$

Then

$$\begin{array}{r}{\mathbf{U}}^T\mathbf{U}=\left[\begin{array}{c}{\mathbf{u}}_1^T\\ {\mathbf{u}}_2^T\\ ⋮\\ {\mathbf{u}}_n^T\end{array}\right]\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]=\left[\begin{array}{c}{\mathbf{u}}_i\cdot {\mathbf{u}}_j\end{array}\right]=\mathbf{I}.\end{array}$$

Since columns of $\mathbf{U}$ are linearly independent and span ${\mathbb{R}}^n$, hence $\mathbf{U}$ is invertible. Thus

$${\mathbf{U}}^T={\mathbf{U}}^{-1}.$$

Lemma 4.45 (Determinant of an orthogonal matrix)

Determinant of an orthogonal matrix is $\pm 1$.

Proof. Let $\mathbf{U}$ be an orthogonal matrix. Then

$$det({\mathbf{U}}^T\mathbf{U})=det(\mathbf{I})⟹{(det(\mathbf{U}))}^2=1.$$

Thus we have

$$det(\mathbf{U})=\pm 1.$$

4.8.2. Unitary Matrices

Definition 4.110 (Unitary matrix)

A complex square matrix $\mathbf{U}$ is called unitary if the columns of $\mathbf{U}$ form an orthonormal set. In other words, let

$$\mathbf{U}=\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]$$

with $u_i\in {\mathbb{C}}^n$. Then we have

$${\mathbf{u}}_i\cdot {\mathbf{u}}_j=⟨{\mathbf{u}}_i,{\mathbf{u}}_j⟩={\mathbf{u}}_j^H{\mathbf{u}}_i={\delta }_{i,j}.$$

Lemma 4.46

A unitary matrix $\mathbf{U}$ is invertible with ${\mathbf{U}}^H={\mathbf{U}}^{-1}$.

Proof. Let

$$\mathbf{U}=\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]$$

be unitary with

$$\begin{array}{r}{\mathbf{U}}^H=\left[\begin{array}{c}{\mathbf{u}}_1^H\\ {\mathbf{u}}_2^H\\ ⋮\\ {\mathbf{u}}_n^H\end{array}\right].\end{array}$$

Then

$$\begin{array}{r}{\mathbf{U}}^H\mathbf{U}=\left[\begin{array}{c}{\mathbf{u}}_1^H\\ {\mathbf{u}}_2^H\\ ⋮\\ {\mathbf{u}}_n^H\end{array}\right]\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]=\left[\begin{array}{c}{\mathbf{u}}_i^H{\mathbf{u}}_j\end{array}\right]=\mathbf{I}.\end{array}$$

Since columns of $\mathbf{U}$ are linearly independent and span ${\mathbb{C}}^n$, hence $\mathbf{U}$ is invertible. Thus

$${\mathbf{U}}^H={\mathbf{U}}^{-1}.$$

Lemma 4.47 (Determinant of unitary matrices)

The magnitude of determinant of a unitary matrix is $1$.

Proof. Let $\mathbf{U}$ be a unitary matrix. Then

$$det({\mathbf{U}}^H\mathbf{U})=det(\mathbf{I})⟹det({\mathbf{U}}^H)det(\mathbf{U})=1⟹\overline{det(\mathbf{U})}det(\mathbf{U})=1.$$

Thus we have

$$|det(\mathbf{U}){|}^2=1⟹|det(\mathbf{U})|=1.$$

4.8.3. F Unitary Matrices

We provide a common definition for unitary matrices over any field $\mathbb{F}$. This definition applies to both real and complex matrices.

Definition 4.111

A square matrix $\mathbf{U}\in {\mathbb{F}}^{n\times n}$ is called $\mathbb{F}$ unitary if the columns of $\mathbf{U}$ form an orthonormal set. In other words, let

$$\mathbf{U}=\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& \dots & {\mathbf{u}}_n\end{array}\right]$$

with ${\mathbf{u}}_i\in {\mathbb{F}}^n$. Then we have

$$⟨{\mathbf{u}}_i,{\mathbf{u}}_j⟩={\mathbf{u}}_j^H{\mathbf{u}}_i={\delta }_{i,j}.$$

We note that a suitable definition of inner product transports the definition appropriately into orthogonal matrices over $\mathbb{R}$ and unitary matrices over $\mathbb{C}$.

When we are talking about $\mathbb{F}$ unitary matrices, then we will use the symbol ${\mathbf{U}}^H$ to mean its inverse. In the complex case, it will map to its conjugate transpose, while in real case it will map to simple transpose.

This definition helps us simplify some of the discussions in the sequel (like singular value decomposition).

Following results apply equally to orthogonal matrices for real case and unitary matrices for complex case.

Lemma 4.48 (Norm preservation)

$\mathbb{F}$-unitary matrices preserve norm. i.e.

$$\Vert \mathbf{U}\mathbf{x}{\Vert }_2=\Vert \mathbf{x}{\Vert }_2.$$

Proof. We have

$$\Vert \mathbf{U}\mathbf{x}{\Vert }_2^2=(\mathbf{U}\mathbf{x}{)}^H(\mathbf{U}\mathbf{x})={\mathbf{x}}^H{\mathbf{U}}^H\mathbf{U}\mathbf{x}={\mathbf{x}}^H\mathbf{I}\mathbf{x}=\Vert \mathbf{x}{\Vert }_2^2.$$

Remark 4.11

For the real case we have

$$\Vert \mathbf{U}\mathbf{x}{\Vert }_2^2=(\mathbf{U}\mathbf{x}{)}^T(\mathbf{U}\mathbf{x})={\mathbf{x}}^T{\mathbf{U}}^T\mathbf{U}\mathbf{x}={\mathbf{x}}^T\mathbf{I}\mathbf{x}=\Vert \mathbf{x}{\Vert }_2^2.$$

Lemma 4.49 (Preservation of inner products)

$\mathbb{F}$-unitary matrices preserve inner product. i.e.

$$⟨\mathbf{U}\mathbf{x},\mathbf{U}\mathbf{y}⟩=⟨\mathbf{x},\mathbf{y}⟩.$$

Proof. We have

$$⟨\mathbf{U}\mathbf{x},\mathbf{U}\mathbf{y}⟩=(\mathbf{U}\mathbf{y}{)}^H\mathbf{U}\mathbf{x}={\mathbf{y}}^H{\mathbf{U}}^H\mathbf{U}\mathbf{x}={\mathbf{y}}^H\mathbf{x}.$$

Remark 4.12

For the real case we have

$$⟨\mathbf{U}\mathbf{x},\mathbf{U}\mathbf{y}⟩=(\mathbf{U}\mathbf{y}{)}^T\mathbf{U}\mathbf{x}={\mathbf{y}}^T{\mathbf{U}}^T\mathbf{U}\mathbf{x}={\mathbf{y}}^T\mathbf{x}.$$