p.204, Definition 7.2, Adjoint $\adjt{T}$
Suppose $T\in\linmap{V}{W}$. Fix $w\in W$. Define $S_w:W\mapsto\wF$ by $S_w(\cdot)\equiv\innprd{\cdot}{w}$. Then 6.7(a) shows that $S_w$ is a linear map. Since the product (i.e. composition) of linear maps is linear (by W.3.21), then $S_wT=S_w\circ T\in\linmap{V}{\wF}$.
Define the linear functional $\varphi_{T,w}\in\linmap{V}{\wF}$ by $\varphi_{T,w}\equiv S_wT$. This linear functional depends on $T$ and $w$:
$$ \varphi_{T,w}(v)\equiv (S_wT)v=S_w(Tv)=\innprd{Tv}{w} $$
By the Riesz Representation Theorem (6.42), there exists a unique vector in $V$, call it $\adjt{T}w$, such that this linear functional is given by taking the inner product with it. That is, there exists a unique vector $\adjt{T}w\in V$ such that, for every $v\in V$, we have
$$ \innprd{Tv}{w}_W=\varphi_{T,w}(v)=\innprd{v}{\adjt{T}w}_V \tag{W.7.Adj.1} $$
Since $\varphi_{T,w}$ depends on $T$ and $w$, then $\adjt{T}$ depends on $w$. That is, $\adjt{T}$ is a function of $w$. And we can show that $\adjt{T}$ is unique. Suppose there exists another function $X:W\mapsto V$ that satisfies W.7.Adj.1 but with $Xz\neq \adjt{T}z$ for some $z\in W$. Then
$$ \innprd{v}{Xz}_V=\varphi_{T,z}(v)=\innprd{v}{\adjt{T}z}_V\quad\quad Xz\neq \adjt{T}z $$
But this contradicts the Riesz implication of the existence of the unique $\adjt{T}z\in V$ satisfying this equality.
Axler (7.5) also shows that $\adjt{T}\in\linmap{W}{V}$. Hence we have established that there exists a unique linear map $\adjt{T}$ from $W$ to $V$ satisfying
$$ \innprd{Tv}{w}_W=\innprd{v}{\adjt{T}w}_V \tag{W.7.Adj.2} $$
for every $v\in V$ and every $w\in W$.
Computation of Adjoint
Let $e_1,\dots,e_n$ be an orthonormal basis of $V$. Then 6.43 gives
$$ \adjt{T}w=\sum_{k=1}^n\cj{\varphi_{T,w}(e_k)}e_k=\sum_{k=1}^n\cj{\innprd{Te_k}{w}}e_k=\sum_{k=1}^n\innprd{w}{Te_k}e_k \tag{W.7.CoA.1} $$
Since $\adjt{T}w\in V$, this equality can also be derived from 6.30 and W.7.G.11:
$$ \adjt{T}w=\sum_{k=1}^n\innprd{\adjt{T}w}{e_k}e_k=\sum_{k=1}^n\innprd{w}{Te_k}e_k $$
Suppose $T\in\oper{V}$ is an operator on $V$. Then
$$\align{ \adjt{T}w &= \sum_{k=1}^n\innprd{w}{Te_k}e_k \\ &= \sum_{k=1}^ne_k\innprd{w}{Te_k} \\ &= \sum_{k=1}^ne_k\innprdBg{\sum_{j=1}^n\innprd{w}{e_j}e_j}{Te_k} \\ &= \sum_{k=1}^ne_k\sum_{j=1}^n\innprdbg{\innprd{w}{e_j}e_j}{Te_k} \\ &= \sum_{k=1}^ne_k\sum_{j=1}^n\innprd{w}{e_j}\innprd{e_j}{Te_k}\tag{W.7.CoA.2} \\ &= \sum_{k=1}^ne_k\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_k}{e_j}}\tag{W.7.CoA.3} }$$
For example, define $T\in\oper{V}$ to be a projection down to the first two dimensions of $V$ and also make $T$ a clockwise rotation on the first two dimensions. That is, $T$ is defined by
$$ Te_1=-e_2 \quad\quad\quad Te_2=e_1 \quad\quad\quad Te_k=0\quad\text{for }k=3,\dots,n $$
Note that $\innprd{Te_k}{e_j}=0$ for $k=3,\dots,n$ and for $j=1,\dots,n$. Hence for $k=3,\dots,n$, we get
$$ e_k\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_k}{e_j}} = e_k\sum_{j=1}^n\innprd{w}{e_j}\cj{0} = e_k\cdot0 = 0 \\ $$
We also have $\innprd{Te_1}{e_j}=0$ for $j\neq2$ and $\innprd{Te_1}{e_2}=-1$. Hence
$$ e_1\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_1}{e_j}} = \innprd{w}{e_2}\cj{\innprd{Te_1}{e_2}}e_1=-\innprd{w}{e_2}e_1 $$
And we have $\innprd{Te_2}{e_j}=0$ for $j\neq1$ and $\innprd{Te_2}{e_1}=1$. Hence
$$ e_2\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_2}{e_j}} = \innprd{w}{e_1}\cj{\innprd{Te_2}{e_1}}e_2=\innprd{w}{e_1}e_2 $$
Hence, formula W.7.CoA.3 gives
$$\align{ \adjt{T}w &= \sum_{k=1}^ne_k\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_k}{e_j}} \\ &= e_1\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_1}{e_j}}+e_2\sum_{j=1}^n\innprd{w}{e_j}\cj{\innprd{Te_2}{e_j}} \\ &= -\innprd{w}{e_2}e_1+\innprd{w}{e_1}e_2 }$$
or formula W.7.CoA.1 gives
$$\align{ \adjt{T}w &= \sum_{k=1}^n\innprd{w}{Te_k}e_k \\ &= \innprd{w}{Te_1}e_1+\innprd{w}{Te_2}e_2+\sum_{k=3}^n\innprd{w}{0}e_k \\ &= -\innprd{w}{e_2}e_1+\innprd{w}{e_1}e_2 \\ &= -\innprdBg{\sum_{k=1}^n\innprd{w}{e_k}e_k}{e_2}e_1+\innprdBg{\sum_{k=1}^n\innprd{w}{e_k}e_k}{e_1}e_2 \\ &= -\sum_{k=1}^n\innprdbg{\innprd{w}{e_k}e_k}{e_2}e_1+\sum_{k=1}^n\innprdbg{\innprd{w}{e_k}e_k}{e_1}e_2 \\ &= -\sum_{k=1}^n\innprd{w}{e_k}\innprd{e_k}{e_2}e_1+\sum_{k=1}^n\innprd{w}{e_k}\innprd{e_k}{e_1}e_2 \\ &= -\innprd{w}{e_2}e_1+\innprd{w}{e_1}e_2 }$$
whereas
$$\align{ Tw &= T\Prn{\sum_{k=1}^n\innprd{w}{e_k}e_k} \\ &= \sum_{k=1}^n\innprd{w}{e_k}Te_k \\ &= \innprd{w}{e_1}Te_1+\innprd{w}{e_2}Te_2 \\ &= -\innprd{w}{e_1}e_2+\innprd{w}{e_2}e_1 \\ &= \innprd{w}{e_2}e_1-\innprd{w}{e_1}e_2 }$$
Hence $T\neq \adjt{T}$ and $T$ is not self-adjoint. But $T$ is normal. We’ll show that in a moment. First let’s show that $\adjt{T}$ is also a projection down to the first two dimensions of $V$ but that it’s a counter-clockwise rotation (whereas $T$ is clockwise):
$$\align{ &\adjt{T}e_1=-\innprd{e_1}{e_2}e_1+\innprd{e_1}{e_1}e_2=-0\cdot e_1+1\cdot e_2=e_2 \\ &\adjt{T}e_2=-\innprd{e_2}{e_2}e_1+\innprd{e_2}{e_1}e_2=-1\cdot e_1+0\cdot e_2=-e_1 \\ &\adjt{T}e_k=-\innprd{e_k}{e_2}e_1+\innprd{e_k}{e_1}e_2=0\cdot e_1+0\cdot e_2=0\quad\text{ for }k=3,\dots,n }$$
And now we show that $T$ is normal:
$$\align{ \tat w &= \adjt{T}\prn{\innprd{w}{e_2}e_1-\innprd{w}{e_1}e_2} \\ &= \innprd{w}{e_2}\adjt{T}e_1-\innprd{w}{e_1}\adjt{T}e_2 \\ &= \innprd{w}{e_2}e_2-\innprd{w}{e_1}(-e_1) \\ &= \innprd{w}{e_2}e_2+\innprd{w}{e_1}e_1 \\ &= \innprd{w}{e_1}e_1+\innprd{w}{e_2}e_2 }$$
and
$$\align{ \tta w &= T\prn{-\innprd{w}{e_2}e_1+\innprd{w}{e_1}e_2} \\ &= -\innprd{w}{e_2}Te_1+\innprd{w}{e_1}Te_2 \\ &= -\innprd{w}{e_2}(-e_2)+\innprd{w}{e_1}e_1 \\ &= \innprd{w}{e_2}e_2+\innprd{w}{e_1}e_1 \\ &= \innprd{w}{e_1}e_1+\innprd{w}{e_2}e_2 \\ &= \tat w }$$
Next we’ll show that $\tta=\tat$ is also a projection down to the first two dimensions of $V$. But there is no rotation. In fact, we’ll show that $\tat=(\tat)_{\span{e_1,e_2}}$ is an orthogonal projection.
$$\align{ &\tat e_1=\innprd{e_1}{e_1}e_1+\innprd{e_1}{e_2}e_2=1\cdot e_1+0\cdot e_2=e_1 \\ &\tat e_2=\innprd{e_2}{e_1}e_1+\innprd{e_2}{e_2}e_2=0\cdot e_1+1\cdot e_2=e_2 \\ &\tat e_k=\innprd{e_k}{e_1}e_1+\innprd{e_k}{e_2}e_2=0\cdot e_1+0\cdot e_2=0\quad\text{ for }k=3,\dots,n }$$
To show that $\tat$ is an orthogonal projection onto ${\span{e_1,e_2}}$, it suffices to show that $\rangsp{\tat}={\span{e_1,e_2}}$, that $\tat=(\tat)^2$, and that $\nullsp{\tat}\subset\rangsp{\tat}^\perp$ (by exercise 6.C.7). Proposition W.3.10 and the above equations give
$$ \rangsp{\tat}=\span{\tat e_1,\tat e_2,\tat e_3,\dots,\tat e_n}=\span{e_1,e_2,0,\dots,0}=\span{e_1,e_2} $$
The above equations also give
$$\align{ (\tat)^2w &= \tat(\tat w) \\ &= \tat\prn{\innprd{w}{e_1}e_1+\innprd{w}{e_2}e_2} \\ &= \innprd{w}{e_1}\tat e_1+\innprd{w}{e_2}\tat e_2 \\ &= \innprd{w}{e_1}e_1+\innprd{w}{e_2}e_2 \\ &= \tat w }$$
Let $u\in\nullsp{\tat}$ and let $z\in\rangsp{\tat}$. Then $0=\tat u$ and $z=\tat v$ for some $v\in V$. Hence
$$\align{ \innprd{u}{z} &= \innprd{u}{\tat v} \\ &= \innprd{Tu}{Tv}\tag{by W.7.Adj.2} \\ &= \innprd{Tu}{(\adjt{T})^*v}\tag{by 7.6.c} \\ &= \innprd{\tat u}{v}\tag{by W.7.Adj.2} \\ &= \innprd{0}{v} \\ &= 0\tag{by 6.7.b} }$$
Hence $\nullsp{\tat}\subset\rangsp{\tat}^\perp$.
Rather than proving that $\nullsp{\tat}\subset\rangsp{\tat}^\perp$, we can alternatively prove that $\tat$ is self-adjoint (by exercise 7.A.11). We have
$$ (\tat)^*=\adjt{T}(\adjt{T})^*=\tat $$
The first equality holds by 7.6(e) and the second equality holds by 7.6(c). Notice that the equality $(\tat)^*=\tat$ holds for all linear maps $T$. Hence, if $T$ satisfies $\tat=(\tat)^2$, then $\tat=(\tat)_{\rangsp{\tat}}$ is an orthogonal projection into $\rangsp{\tat}$.
Verification of Adjoint Computation
Let $e_1,\dots,e_n$ be an orthonormal basis of $V$. Formula W.7.CofA.1 from the previous section is
$$ \adjt{T}w=\sum_{k=1}^n\cj{\innprd{Te_k}{w}}e_k=\sum_{k=1}^n\innprd{w}{Te_k}e_k $$
We wish to show directly that this formula for $\adjt{T}w$ satisfies the equality from W.7.Adj.2:
$$ \innprd{Tv}{w}_W=\innprd{v}{\adjt{T}w}_V $$
for all $v\in V$ and all $w\in W$. We have
$$\align{ \innprd{v}{\adjt{T}w}_V &= \innprdBgg{v}{\sum_{k=1}^n\cj{\innprd{Te_k}{w}_W}e_k}_V \\ &= \sum_{k=1}^n\innprdbg{v}{\cj{\innprd{Te_k}{w}_W}e_k}_V\tag{by 6.7.d} \\ &= \sum_{k=1}^n\innprd{Te_k}{w}_W\innprd{v}{e_k}_V\tag{by 6.7.e} \\ &= \sum_{k=1}^n\innprdbg{\innprd{v}{e_k}_VTe_k}{w}_W\tag{by 6.3.homogeneity} \\ &= \innprdBgg{\sum_{k=1}^n\innprd{v}{e_k}_VTe_k}{w}_W\tag{by additivity in first slot} \\ &= \innprdBgg{T\Prn{\sum_{k=1}^n\innprd{v}{e_k}_Ve_k}}{w}_W\tag{by 3.2} \\ &= \innprd{Tv}{w}_W\tag{by 6.30} }$$
General
Proposition W.7.G.1 (7.7, p.207) Suppose $T\in\linmap{V}{W}$. Let $w\in W$. Then $\adjt{T}w=0$ if and only if $\innprd{v}{\adjt{T}w}=0$ for all $v\in V$.
Proof Suppose $\adjt{T}w=0$. Then 6.7(b) implies that $\innprd{v}{\adjt{T}w}=0$. Conversely, suppose $\innprd{v}{\adjt{T}w}=0$ for all $v\in V$. By definition, $\adjt{T}w\in V$. Hence $\innprd{\adjt{T}w}{\adjt{T}w}=0$ and the definiteness property of an inner product implies that $\adjt{T}w=0$. $\blacksquare$
Proposition W.7.G.2 (7.11, p.209) $T\in\oper{V}$ is self-adjoint if and only if $\innprd{Tv}{w}=\innprd{v}{Tw}$ for all $v,w\in V$.
Proof Suppose $T$ is self-adjoint. Then $T=\adjt{T}$ and
$$ \innprd{Tv}{w}=\innprd{v}{\adjt{T}w}=\innprd{v}{Tw} $$
for all $v,w\in V$. The first equality follows from the defintion (7.2) of $\adjt{T}$.
Suppose $\innprd{Tv}{w}=\innprd{v}{Tw}$ for all $v,w\in V$. Then
$$ \innprd{v}{\adjt{T}w}=\innprd{Tv}{w}=\innprd{v}{Tw} $$
for all $v,w\in V$. The first equality follows from the defintion (7.2) of $\adjt{T}$. Hence
$$ 0=\innprd{v}{\adjt{T}w}-\innprd{v}{Tw}=\innprd{v}{\adjt{T}w-Tw}=\innprd{v}{(\adjt{T}-T)w} $$
for all $v,w\in V$. In particular, take $v=(\adjt{T}-T)w$. Then $0=\innprdbg{(\adjt{T}-T)w}{(\adjt{T}-T)w}$ and the definiteness property of an inner product implies that $(\adjt{T}-T)w=0$. Since this is true for any $w\in V$, then $\adjt{T}-T=0$. $\blacksquare$
Proposition W.7.G.3 (p.209) The sum of two self-adjoint operators is self-adjoint. Also, the product of a real scalar and a self-adjoint operator is self-adjoint.
Proof Let $S$ and $T$ be self-adjoint operators. Then
$$\align{ \innprd{(S+T)v}{w} &= \innprd{Sv+Tv}{w} \\ &= \innprd{Sv}{w}+\innprd{Tv}{w} \\ &= \innprd{v}{Sw}+\innprd{v}{Tw} \\ &= \innprd{v}{Sw+Tw} \\ &= \innprd{v}{(S+T)w} }$$
Let $T$ be self-adjoint and let $\lambda\in\wR$. Then
$$\align{ \innprd{(\lambda T)v}{w} &= \innprd{\lambda (Tv)}{w} \\ &= \lambda \innprd{Tv}{w} \\ &= \lambda \innprd{v}{Tw} \\ &= \innprd{v}{\overline{\lambda}(Tw)} \\ &= \innprd{v}{\lambda(Tw)} \\ &= \innprd{v}{(\lambda T)w} \quad\blacksquare }$$
Proposition 7.16 p.211 The text gives $\innprd{Tw}{u}=\innprd{Tu}{w}$. This gives 7.16.1 below:
$$\align{ \frac{\innprd{T(u+w)}{u+w}-\innprd{T(u-w)}{u-w}}4 &= \frac{\innprd{Tu+Tw}{u+w}-\innprd{Tu-Tw}{u-w}}4 \\ &= \frac{\innprd{Tu+Tw}{u+w}}4 \\ &- \frac{\innprd{Tu-Tw}{u-w}}4 \\ &= \frac{\innprd{Tu}{u}+\innprd{Tu}{w}+\innprd{Tw}{u}+\innprd{Tw}{w}}4 \\ &- \frac{\innprd{Tu}{u}-\innprd{Tu}{w}-\innprd{Tw}{u}+\innprd{Tw}{w}}4 \\ &= \frac{\innprd{Tu}{w}+\innprd{Tw}{u}}4-\frac{-\innprd{Tu}{w}-\innprd{Tw}{u}}4 \\ &= \frac{\innprd{Tu}{w}+\innprd{Tw}{u}}4+\frac{+\innprd{Tu}{w}+\innprd{Tw}{u}}4 \\ &= \frac{\innprd{Tu}{w}+\innprd{Tw}{u}+\innprd{Tu}{w}+\innprd{Tw}{u}}4 \\ &= \frac{2\innprd{Tu}{w}+2\innprd{Tw}{u}}4 \\ &= \frac{2\innprd{Tu}{w}+2\innprd{Tu}{w}}4\tag{7.16.1} \\ &= \frac{4\innprd{Tu}{w}}4 \\ &= \innprd{Tu}{w} \quad\blacksquare }$$
Example 7.19, p.212 Show that $T$ is not self-adjoint.
Solution Proposition 7.10 gives
$$ \mtrxof{\adjt{T},(e_1,e_2)}=\overline{\mtrxof{T,(e_1,e_2)}^t}=\overline{\pmtrx{2&-3\\3&2}^t}=\overline{\pmtrx{2&3\\-3&2}}=\pmtrx{2&3\\-3&2} $$
Suppose $T$ is self-adjoint. Then $T=\adjt{T}$ and
$$ \pmtrx{2&-3\\3&2}=\mtrxof{T,(e_1,e_2)}=\mtrxof{\adjt{T},(e_1,e_2)}=\pmtrx{2&3\\-3&2} $$
Contradiction, hence $T$ is not self-adjoint. $\blacksquare$
Proposition W.7.G.4 (7.20, p.213) Suppose $T\in\oper{V}$. Then $\tat-\tta$ is self-adjoint.
Proof We have
$$\align{ (\tat-\tta)^* &= (\tat)^*-(\tta)^*\tag{by 7.6.a} \\ &= \adjt{T}(\adjt{T})^*-(\adjt{T})^*\adjt{T}\tag{by 7.6.e} \\ &= \tat-\tta\tag{by 7.6.c}\quad\blacksquare \\ }$$
Proposition W.7.G.5 (7.21, p.213) Suppose $T\in\oper{V}$ is normal and $\lambda\in\wF$. Then $T-\lambda I$ is also normal.
Proof We have
$$\align{ (T-\lambda I)(T-\lambda I)^* &= (T-\lambda I)\prn{\adjt{T}-(\lambda I)^*}\tag{by 7.6.a} \\ &= (T-\lambda I)(\adjt{T}-\overline{\lambda}I^*)\tag{by 7.6.b} \\ &= (T-\lambda I)(\adjt{T}-\overline{\lambda}I)\tag{by 7.6.d} \\ &= \tta-T\overline{\lambda}I-\lambda I\adjt{T}+\lambda I\overline{\lambda}I\tag{by W.3.20.3} \\ &= \tat-T\overline{\lambda}I-\lambda I\adjt{T}+\lambda I\overline{\lambda}I\tag{by hypothesis} \\ &= \tat-\lambda I\adjt{T}-T\overline{\lambda}I+\lambda \overline{\lambda}II \\ &= \tat-\lambda \adjt{T}I-\overline{\lambda}TI+\overline{\lambda}\lambda II\tag{by 3.9.identity} \\ &= \tat-\adjt{T}\lambda I-\overline{\lambda}IT+\overline{\lambda}I\lambda I\tag{by 3.9.identity} \\ &= (\adjt{T}-\overline{\lambda}I)(T-\lambda I)\tag{by W.3.20.3} \\ &= (\adjt{T}-\overline{\lambda}I^*)(T-\lambda I)\tag{by 7.6.d} \\ &= \prn{\adjt{T}-(\lambda I)^*}(T-\lambda I)\tag{by 7.6.b} \\ &= (T-\lambda I)^*(T-\lambda I)\tag{by 7.6.a}\quad\blacksquare \\ }$$
Example W.7.G.6 The Four Fundamental Subspaces 1 Define $T\in\oper{\wR^3}$ by
$$ Te_1\equiv e_1 \quad\quad\quad Te_2\equiv e_2 \quad\quad\quad Te_3\equiv 0 $$
Then
$$ T(x,y,z)=T(xe_1+ye_2+ze_3)=xTe_1+yTe_2+zTe_3=xe_1+ye_2=(x,y,0) $$
Now let’s see what $\adjt{T}$ looks like
$$\align{ \innprdbg{(a,b,c)}{\adjt{T}(x,y,z)} &= \innprdbg{T(a,b,c)}{(x,y,z)} \\ &= \innprdbg{(a,b,0)}{(x,y,z)} \\ &= ax+by \\ &= \innprdbg{(a,b,c)}{(x,y,0)} \\ }$$
Hence $T=\adjt{T}$ and $T$ is self-adjoint. Let’s check out it’s matrix:
$$ \mtrxof{T}_{:,1}=\mtrxof{Te_1}=\mtrxof{e_1}=\pmtrx{1\\0\\0} $$
$$ \mtrxof{T}_{:,2}=\mtrxof{Te_2}=\mtrxof{e_2}=\pmtrx{0\\1\\0} $$
$$ \mtrxof{T}_{:,3}=\mtrxof{Te_3}=\mtrxof{0}=\pmtrx{0\\0\\0} $$
Hence
$$ \mtrxof{T}=\pmtrx{1&0&0\\0&1&0\\0&0&0} $$
And let’s verify that the matrix of $\adjt{T}$ is the same:
$$ \mtrxof{\adjt{T}}=\cj{\mtrxof{T}^t}=\cj{\pmtrx{1&0&0\\0&1&0\\0&0&0}^t}=\pmtrx{1&0&0\\0&1&0\\0&0&0}=\mtrxof{T} $$
And let’s check the four fundamental subspaces from 7.7. First we can see that
$$ \nullsp{T}=\nullsp{\adjt{T}}=\setb{(0,0,z)\in\wR^3:z\in\wR} $$
$$ \rangsp{T}=\rangsp{\adjt{T}}=\setb{(x,y,0)\in\wR^3:x,y\in\wR} $$
Let’s confirm 7.7.a
$$\align{ \rangsp{T}^\perp &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=\innprdbg{(a,b,c)}{(x,y,0)},\forall x,y\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=ax+by+c\cdot0,\forall x,y\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=ax+by,\forall x,y\in\wR} \\ &= \setb{(0,0,c)\in\wR^3:c\in\wR} \\ &= \nullsp{T} \\ &= \nullsp{\adjt{T}}\tag{W.7.G.6.1} }$$
And let’s confirm 7.7.b
$$\align{ \nullsp{T}^\perp &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=\innprdbg{(a,b,c)}{(0,0,z)},\forall z\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=a\cdot0+b\cdot0+cz,\forall z\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=cz,\forall z\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b\in\wR,c=0} \\ &= \setb{(a,b,0)\in\wR^3:a,b\in\wR} \\ &= \rangsp{T} \\ &= \rangsp{\adjt{T}}\tag{W.7.G.6.2} }$$
And let’s confirm 7.7.c:
$$ \rangsp{\adjt{T}}^\perp=(\nullsp{T}^\perp)^\perp=\nullsp{T} $$
where the first equality follows from W.7.G.6.2.
And let’s confirm 7.7.d:
$$ \nullsp{\adjt{T}}^\perp=(\rangsp{T}^\perp)^\perp=\rangsp{T} $$
where the first equality follows from W.7.G.6.1.
Example W.7.G.7 The Four Fundamental Subspaces 2 In example W.6.G.4, we examined the tilt map. Let’s look at this again. Define $T\in\oper{\wR^3}$ by
$$ Te_1\equiv e_1-e_3 \quad\quad\quad Te_2\equiv e_2 \quad\quad\quad Te_3\equiv 0 $$
Then
$$\align{ T(x,y,z) &= T(xe_1+ye_2+ze_3) \\ &= xTe_1+yTe_2+zTe_3 \\ &= x(e_1-e_3)+ye_2 \\ &= xe_1+ye_2-xe_3 \\ &= (x,y,-x) }$$
Now let’s see what $\adjt{T}$ looks like
$$\align{ \innprdbg{(a,b,c)}{\adjt{T}(x,y,z)} &= \innprdbg{T(a,b,c)}{(x,y,z)} \\ &= \innprdbg{(a,b,-a)}{(x,y,z)} \\ &= ax+by-az \\ &= \innprdbg{(a,b,c)}{(x-z,y,0)} \\ }$$
Hence $T\neq \adjt{T}$ and $T$ is not self-adjoint. Let’s check out it’s matrix:
$$\align{ &\mtrxofb{T,(e_1,e_2,e_3)}_{:,1}=\mtrxofb{Te_1,(e_1,e_2,e_3)}=\mtrxofb{e_1-e_3,(e_1,e_2,e_3)}=\pmtrx{1\\0\\-1} \\ &\mtrxofb{T,(e_1,e_2,e_3)}_{:,2}=\mtrxofb{Te_2,(e_1,e_2,e_3)}=\mtrxofb{e_2,(e_1,e_2,e_3)}=\pmtrx{0\\1\\0} \\ &\mtrxofb{T,(e_1,e_2,e_3)}_{:,3}=\mtrxofb{Te_3,(e_1,e_2,e_3)}=\mtrxofb{0,(e_1,e_2,e_3)}=\pmtrx{0\\0\\0} }$$
Hence
$$ \mtrxofb{T,(e_1,e_2,e_3)}=\pmtrx{1&0&0\\0&1&0\\-1&0&0} \tag{W.7.G.7.1} $$
and
$$ \mtrxof{\adjt{T}}=\cj{\mtrxof{T}^t}=\cj{\pmtrx{1&0&0\\0&1&0\\-1&0&0}^t}=\cj{\pmtrx{1&0&-1\\0&1&0\\0&0&0}}=\pmtrx{1&0&-1\\0&1&0\\0&0&0}\neq\mtrxof{T} $$
And let’s check the four fundamental subspaces from 7.7. First we can see that
$$ \nullsp{T}=\setb{(0,0,z)\in\wR^3:z\in\wR} $$
$$ \rangsp{T}=\setb{(x,y,-x)\in\wR^3:x,y\in\wR} $$
$$\align{ \nullsp{\adjt{T}} &= \setb{(x,y,z)\in\wR^3:0=x−z,0=y} \\ &= \setb{(x,0,z)\in\wR^3:0=x−z} \\ &= \setb{(x,0,z)\in\wR^3:x=z} \\ &= \setb{(x,0,x)\in\wR^3:x\in\wR} \\ }$$
$$\align{ \rangsp{\adjt{T}} &= \setb{(x−z,y,0)\in\wR^3:x,y,z\in\wR} \\ &= \setb{(x,y,0)\in\wR^3:x,y\in\wR} }$$
Let’s confirm 7.7.a
$$\align{ \rangsp{T}^\perp &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=\innprdbg{(a,b,c)}{(x,y,-x)},\forall x,y\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=ax+by-cx,\forall x,y\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=(a-c)x+by,\forall x,y\in\wR} \\ &= \setb{(a,0,c)\in\wR^3:0=a-c} \\ &= \setb{(a,0,c)\in\wR^3:a=c} \\ &= \setb{(a,0,a)\in\wR^3:a\in\wR} \\ &= \nullsp{\adjt{T}} }$$
And let’s confirm 7.7.b
$$\align{ \nullsp{T}^\perp &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=\innprdbg{(a,b,c)}{(0,0,z)},\forall z\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=a\cdot0+b\cdot0+cz,\forall z\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b,c\in\wR\text{ and }0=cz,\forall z\in\wR} \\ &= \setb{(a,b,c)\in\wR^3:a,b\in\wR,c=0} \\ &= \setb{(a,b,0)\in\wR^3:a,b\in\wR} \\ &= \rangsp{\adjt{T}} }$$
And confirmation of 7.7.c and 7.7.d always follow directly from the previous equations:
$$ \rangsp{\adjt{T}}=\nullsp{T}^\perp\quad\implies\quad\rangsp{\adjt{T}}^\perp=(\nullsp{T}^\perp)^\perp=\nullsp{T} $$
$$ \nullsp{\adjt{T}}=\rangsp{T}^\perp\quad\implies\quad\nullsp{\adjt{T}}^\perp=(\rangsp{T}^\perp)^\perp=\rangsp{T} $$
Let’s compute the eigenvalues of $\tat$. Note that
$$ \mtrxof{\adjt{T}T}=\mtrxof{\adjt{T}}\mtrxof{T}=\pmtrx{1&0&-1\\0&1&0\\0&0&0}\pmtrx{1&0&0\\0&1&0\\-1&0&0}=\pmtrx{2&0&0\\0&1&0\\0&0&0} $$
Then 5.32 gives that the eigenvalues of $\tat$ are $2,1,0$. Let’s confirm this. From above, we have $T(x,y,z)=(x,y,-x)$ and $T(x,y,z)=(x-z,y,0)$. Hence
$$ \tat(x,y,z)=\adjtT(x,y,-x)=(x-(-x),y,0)=(2x,y,0) $$
Then the eigenpair equation is
$$ (\lambda x,\lambda y,\lambda z)=\lambda(x,y,z)=(2x,y,0) $$
This becomes the system
$$ \lambda x=2x \\ \lambda y=y \\ \lambda z=0 $$
If $\lambda=0$, then it must be that $x=0$ and $y=0$ while any $z\in\wR$ will work. If $\lambda=1$, then it must be that $z=0$ and $x=0$ while any $y\in\wR$ will work. If $\lambda=2$, then it must be that $z=0$ and $y=0$ while any $x\in\wR$ will work. So we have confirmed that the eigenvalues of $\tat$ are $2,1,0$.
Next let’s look at a Polar Decomposition of the tilt map $T$. From W.7.G.21, we have
$$ \sqrt{\tat}e_1=\sqrt2e_1 \dq\dq \sqrt{\tat}e_2=e_2 \dq\dq \sqrt{\tat}e_3=0e_3 $$
Hence
$$ \sqrt{\tat}(x,y,z)=(\sqrt2x,y,0) $$
and
$$ \rangspb{\sqrt{\tat}}=\setb{(x,y,0)\in\wR^3:x,y\in\wR} $$
and
$$ \rangspb{\sqrttat}^\perp=\setb{(0,0,z)\in\wR^3:z\in\wR} $$
To find an isometry $S$ such that $T=S\sqrt{\tat}$, first let’s find $S_1\in\linmapb{\rangspb{\sqrt{\tat}}}{\rangsp{T}}$ as in the proof of 7.45. Similar to 7.47, for $(x,y,z)\in\wR^3$, we must find $S_1$ that satisfies
$$ S_1(\sqrt2x,y,0)=S_1\sqrt{\tat}(x,y,z)=T(x,y,z)=(x,y,-x) $$
Define $S_1\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ by
$$ S_1(x,y,0)\equiv\Prn{\frac{x}{\sqrt2},y,-\frac{x}{\sqrt2}} $$
or
$$ S_1e_1=\frac1{\sqrt2}e_1-\frac1{\sqrt2}e_3 \dq\dq S_1e_2=e_2 $$
so that
$$ S_1\sqrt{\tat}(x,y,0)=S_1(\sqrt2x,y,0)=\Prn{\frac{\sqrt2x}{\sqrt2},y,-\frac{\sqrt2x}{\sqrt2}}=(x,y,-x)=T(x,y,z) $$
and
$$ S_1(x,y,0)=\Prn{\frac{x}{\sqrt2},y,-\frac{x}{\sqrt2}}\in\setb{(x,y,-x)\in\wR^3:x,y\in\wR}=\rangsp{T} $$
By W.7.G.25(c), we have $\dnorm{S_1(x,y,0)}=\dnorm{(x,y,0)}$ for all $(x,y,0)\in\rangspb{\sqrt{\tat}}=\set{(x,y,0)\in\wR^3:x,y\in\wR}$. Let’s verify that:
$$ \dnorm{S_1(x,y,0)}^2=\dnorm{\Prn{\frac{x}{\sqrt2},y,-\frac{x}{\sqrt2}}}^2=\Prn{\frac{x}{\sqrt2}}^2+y^2+\Prn{-\frac{x}{\sqrt2}}^2=\frac{x^2}2+\frac{x^2}2+y^2=x^2+y^2=\dnorm{(x,y,0)}^2 $$
Now let’s define $f_1\equiv S_1e_1=\Prn{\frac1{\sqrt2},0,-\frac1{\sqrt2}}$ and $f_2\equiv S_1e_2=e_2$. By W.7.G.25(h), $f_1,f_2$ is an orthonormal basis of $\rangsp{T}=\set{(x,y,-x)\in\wR^3:x,y\in\wR}$, which is easy to see geometrically.
Next we would like to extend $f_1,f_2$ to an orthonormal basis of $\wR^3$ in such a way that $f_3$ is an orthonormal basis of $\rangsp{T}^\perp$. The reasons for this will become clear very soon. Since every vector in $\rangsp{T}^\perp$ is orthogonal to every vector in $\rangsp{T}=\span{f_1,f_2}$, then any orthonormal basis of $\rangsp{T}^\perp$ will extend $f_1,f_2$ to an orthonormal basis of $\wR^3=\dirsum{\rangsp{T}}{\rangsp{T}^\perp}$.
So we define $f_3\equiv\Prn{\frac1{\sqrt2},0,\frac1{\sqrt2}}$ to be an orthonormal basis of $\rangsp{T}^\perp=\set{(x,0,x)\in\wR^3:x\in\wR}$. It’s easy to see that $f_1,f_2,f_3$ is an orthonormal basis of $\wR^3$. Now we define $S_2\in\linmapb{\rangspb{\sqrttat}^\perp}{\rangsp{T}^\perp}$ by
$$ S_2e_3\equiv f_3\dq\text{or}\dq S_2(0,0,z)\equiv\Prn{\frac{z}{\sqrt2},0,\frac{z}{\sqrt2}} $$
And let’s check that $S_2$ preserves norms on $\rangspb{\sqrttat}^\perp=\set{(0,0,z)\in\wR^3:z\in\wR}$:
$$ \dnorm{S_2(0,0,z)}^2=\dnorm{\Prn{\frac{z}{\sqrt2},0,\frac{z}{\sqrt2}}}^2=\Prn{\frac{z}{\sqrt2}}^2+\Prn{\frac{z}{\sqrt2}}^2=\frac{z^2}2+\frac{z^2}2=z^2=\dnorm{(0,0,z)}^2 $$
Note that, for $(x,y,z)\in\wR^3$, we have $(x,y,z)=(x,y,0)+(0,0,z)$ so that $(x,y,0)\in\rangspb{\sqrttat}$ and $(0,0,z)\in\rangspb{\sqrttat}^\perp$. Now we define $S\in\oper{\wR^3}$ by
$$ S(x,y,z)\equiv S_1(x,y,0)+S_2(0,0,z)=\Prn{\frac{x}{\sqrt2},y,-\frac{x}{\sqrt2}}+\Prn{\frac{z}{\sqrt2},0,\frac{z}{\sqrt2}}=\Prn{\frac{x}{\sqrt2}+\frac{z}{\sqrt2},y,-\frac{x}{\sqrt2}+\frac{z}{\sqrt2}} $$
Then
$$ S\sqrttat(x,y,z)=S(\sqrt2x,y,0)=S_1(\sqrt2x,y,0)+S_2(0,0,0)=T(x,y,z)+0=T(x,y,z) $$
At the end of the book’s proof of Polar Decomposition, Axler shows that $S$ is an isometry as well. Hence we have completed our Polar Decomposition $T=S\sqrttat$.
Next let’s look at the Singular Value Decomposition of $T$. First recall from above that the eigenvalues of $\tat$ are $2,1,0$ corresponding to the orthonormal eigenbasis $e_1,e_2,e_3$. Hence the singular values of $T$ are $s_1\equiv\sqrt2,s_2\equiv1,s_3\equiv0$ (by W.7.G.22).
Now we come to a key reason why we extended $f_1,f_2$ to an orthonormal basis $f_1,f_2,f_3$ of $\wR^3$ and a key reason why we defined $f_1\equiv S_1e_1$ and $f_2\equiv S_1e_2$ and $S_2e_3\equiv f_3$. Note that
$$\align{ &Se_1=S(1,0,0)=\Prn{\frac{1}{\sqrt2}+\frac{0}{\sqrt2},0,-\frac{1}{\sqrt2}+\frac{0}{\sqrt2}}=\Prn{\frac{1}{\sqrt2},0,-\frac{1}{\sqrt2}}=f_1 \\ &Se_2=S(0,1,0)=\Prn{\frac{0}{\sqrt2}+\frac{0}{\sqrt2},1,-\frac{0}{\sqrt2}+\frac{0}{\sqrt2}}=(0,1,0)=f_2 \\ &Se_3=S(0,0,1)=\Prn{\frac{0}{\sqrt2}+\frac{1}{\sqrt2},0,-\frac{0}{\sqrt2}+\frac{1}{\sqrt2}}=\Prn{\frac{1}{\sqrt2},0,\frac{1}{\sqrt2}}=f_3 }$$
Then the SVD of the tilt map $T$ is
$$\align{ Tv &= s_1\innprd{v}{e_1}Se_1+s_2\innprd{v}{e_2}Se_2+s_3\innprd{v}{e_3}Se_3 \\ &= \sqrt2\innprd{v}{e_1}f_1+1\innprd{v}{e_2}f_2+0\innprd{v}{e_3}f_3 \\ }$$
Hence
$$\align{ Te_1 &= \sqrt2\innprd{e_1}{e_1}f_1+1\innprd{e_1}{e_2}f_2=\sqrt2f_1 \\ Te_2 &= \sqrt2\innprd{e_2}{e_1}f_1+1\innprd{e_2}{e_2}f_2=f_2 \\ Te_3 &= \sqrt2\innprd{e_3}{e_1}f_1+1\innprd{e_3}{e_2}f_2=0=0f_3 }$$
Put $e\equiv e_1,e_2,e_3$ and $e\equiv f_1,f_2,f_3$. Then
$$\align{ \mtrxofb{T,(e_1,e_2,e_3),(f_1,f_2,f_3)} &= \mtrxof{T,e,f} \\ &= \pmtrx{\mtrxof{T,e,f}_{:,1}&\mtrxof{T,e,f}_{:,2}&\mtrxof{T,e,f}_{:,3}} \\ &= \pmtrx{\mtrxof{Te_1,f}&\mtrxof{Te_2,f}&\mtrxof{Te_3,f}} \\ &= \pmtrx{\sqrt2&0&0\\0&1&0\\0&0&0} }$$
Compare this matrix with W.7.G.7.1 from above:
$$ \mtrxofb{T,(e_1,e_2,e_3)}=\pmtrx{1&0&0\\0&1&0\\-1&0&0} $$
Note that this change of basis preserves lengths:
$$\align{ &\dnorm{Te_1}^2=\dnorm{e_1-e_3}^2=\dnorm{e_1}^2+\dnorm{e_3}^2=2 \\ &\dnorm{Te_1}^2=\dnorm{\sqrt2f_1}^2=\normb{\sqrt{2}}^2\dnorm{f_1}^2=2 }$$
From W.7.SVD.2, we have
$$\align{ \mtrxof{T,e} &= \mtrxof{S,e,e}\mtrxofb{\sqrttat,e}\adjt{\mtrxof{I,e,e}} \\ &= \pmtrx{\frac1{\sqrt2}&0&\frac1{\sqrt2}\\0&1&0\\-\frac1{\sqrt2}&0&\frac1{\sqrt2}}\pmtrx{\sqrt2&0&0\\0&1&0\\0&0&0}\mtrxof{\adjt{I},e,e} \\ &= \pmtrx{1&0&0\\0&1&0\\-1&0&0}\mtrxof{I,e,e} \\ &= \pmtrx{1&0&0\\0&1&0\\-1&0&0} }$$
Proposition W.7.G.10 Suppose $T\in\oper{V}$ is normal. Then $\nullsp{T}=\nullsp{\adjt{T}}$.
Proof Proposition 7.20 implies that $\dnorm{Tv}=\dnorm{\adjt{T}v}$ for all $v\in V$. Let $u\in\nullsp{T}$. Then
$$ 0=\dnorm{Tu}=\dnorm{\adjt{T}u} $$
Hence $0=\adjt{T}u$ (by 6.10(a)). Hence $u\in\nullsp{\adjt{T}}$. Hence $\nullsp{T}\subset\nullsp{\adjt{T}}$.
Let $u\in\nullsp{\adjt{T}}$. Then
$$ 0=\dnorm{\adjt{T}u}=\dnorm{Tu} $$
Hence $0=Tu$ (by 6.10(a)). Hence $u\in\nullsp{T}$. Hence $\nullsp{\adjt{T}}\subset\nullsp{T}$ and $\nullsp{\adjt{T}}=\nullsp{T}$. $\blacksquare$
Proposition W.7.G.11 Suppose $T\in\linmap{V}{W}$. Then $\innprd{\adjt{T}w}{v}=\innprd{w}{Tv}$ for every $v\in V$ and every $w\in W$.
Proof $\innprd{\adjt{T}w}{v}=\cj{\innprd{v}{\adjt{T}w}}=\cj{\innprd{Tv}{w}}=\innprd{w}{Tv}$ $\blacksquare$
Example 7.23, p.217 First let’s check that $\frac{(i,1)}{\sqrt2},\frac{(-i,1)}{\sqrt2}$ is an orthonormal basis of $\wC^2$.
$$\align{ \innprdBg{\frac{(i,1)}{\sqrt2}}{\frac{(-i,1)}{\sqrt2}} &= \frac1{\sqrt2}\frac1{\sqrt2}\innprdbg{(i,1)}{(-i,1)} \\ &= \frac12(i\cdot\cj{(-i)}+1\cdot\cj{1})\tag{by W.6.G.9} \\ &= \frac12(i\cdot i+1) \\ &= \frac12(-1+1) \\ &= 0 }$$
$$\align{ \innprdBg{\frac{(i,1)}{\sqrt2}}{\frac{(i,1)}{\sqrt2}} &= \frac1{\sqrt2}\frac1{\sqrt2}\innprdbg{(i,1)}{(i,1)} \\ &= \frac12(i\cdot\cj{i}+1\cdot\cj{1})\tag{by W.6.G.9} \\ &= \frac12(i\cdot(-i)+1) \\ &= \frac12(-i^2+1) \\ &= \frac12(-(-1)+1) \\ &= \frac12(1+1) \\ &= 1 }$$
$$\align{ \innprdBg{\frac{(-i,1)}{\sqrt2}}{\frac{(-i,1)}{\sqrt2}} &= \frac1{\sqrt2}\frac1{\sqrt2}\innprdbg{(-i,1)}{(-i,1)} \\ &= \frac12(-i\cdot\cj{-i}+1\cdot\cj{1})\tag{by W.6.G.9} \\ &= \frac12(-i\cdot i+1) \\ &= \frac12(-i^2+1) \\ &= \frac12(-(-1)+1) \\ &= \frac12(1+1) \\ &= 1 }$$
Now let’s find $T$’s action on the standard basis. Recall that
$$ \mtrxof{T,(e_1,e_2)}\equiv\pmtrx{2&-3\\3&2} $$
Hence
$$ Te_1=2e_1+3e_2 \quad\quad\quad\quad Te_2=-3e_1+2e_2 $$
Now define $f_1\equiv\frac{(i,1)}{\sqrt2}$ and $f_2\equiv\frac{(-i,1)}{\sqrt2}$. Then
$$\align{ Tf_1 &= T\Prn{\frac{(i,1)}{\sqrt2}} \\ &= \frac1{\sqrt2}T(i,1) \\ &= \frac1{\sqrt2}T\prn{(i,0)+(0,1)} \\ &= \frac1{\sqrt2}T\prn{i(1,0)+(0,1)} \\ &= \frac1{\sqrt2}\prn{iT(1,0)+T(0,1)} \\ &= \frac1{\sqrt2}\prn{iTe_1+Te_2} \\ &= \frac1{\sqrt2}\prn{i(2e_1+3e_2)+(-3e_1+2e_2)} \\ &= \frac1{\sqrt2}\prn{i2e_1+i3e_2-3e_1+2e_2} \\ &= \frac1{\sqrt2}\prn{2i(1,0)+3i(0,1)-3(1,0)+2(0,1)} \\ &= \frac1{\sqrt2}\prn{(2i,0)+(0,3i)+(-3,0)+(0,2)} \\ &= \frac1{\sqrt2}\prn{(2i,3i)+(-3,2)} \\ &= \frac1{\sqrt2}(-3+2i,2+3i) \\ &= \frac1{\sqrt2}(3i^2+2i,2+3i) \\ &= \frac1{\sqrt2}(2i+3i^2,2+3i) \\ &= (2+3i)\frac1{\sqrt2}(i,1) \\ &= (2+3i)f_1 }$$
and
$$\align{ Tf_2 &= T\Prn{\frac{(-i,1)}{\sqrt2}} \\ &= \frac1{\sqrt2}T(-i,1) \\ &= \frac1{\sqrt2}T\prn{(-i,0)+(0,1)} \\ &= \frac1{\sqrt2}T\prn{-i(1,0)+(0,1)} \\ &= \frac1{\sqrt2}\prn{-iT(1,0)+T(0,1)} \\ &= \frac1{\sqrt2}\prn{-iTe_1+Te_2} \\ &= \frac1{\sqrt2}\prn{-i(2e_1+3e_2)+(-3e_1+2e_2)} \\ &= \frac1{\sqrt2}\prn{-i2e_1-i3e_2-3e_1+2e_2} \\ &= \frac1{\sqrt2}\prn{-2i(1,0)-3i(0,1)-3(1,0)+2(0,1)} \\ &= \frac1{\sqrt2}\prn{(-2i,0)+(0,-3i)+(-3,0)+(0,2)} \\ &= \frac1{\sqrt2}\prn{(-2i,-3i)+(-3,2)} \\ &= \frac1{\sqrt2}(-3-2i,2-3i) \\ &= \frac1{\sqrt2}(3i^2-2i,2-3i) \\ &= \frac1{\sqrt2}(-2i+3i^2,2-3i) \\ &= (2-3i)\frac1{\sqrt2}(-i,1) \\ &= (2-3i)f_2 }$$
Succintly, we have $Tf_1=(2+3i)f_1$ and $Tf_2=(2-3i)f_2$. Hence
$$ \mtrxofb{T,(f_1,f_2)}=\mtrxofb{T,(f_1,f_2),(f_1,f_2)}=\pmtrx{2+3i&0\\0&2-3i} $$
$\blacksquare$
Proposition 7.29 “Suppose $\wF=\wR$ and $T\in\oper{V}$. Then the following are equivalent:
(a) $T$ is self-adjoint.
(b) $V$ has an orthonormal basis consisting of eigenvectors of $T$.
We will prove that (a) implies (b) by induction on $\dim{V}$. To get started, note that if $\dim{V}=1$, then (a) implies (b).”
I think Axler is invoking 7.27: Since $\dim{V}=1$, then $V\neq\set{0}$. Since $T$ is self-adjoint, then we can invoke 7.27 to deduce that $T$ has an eigenvalue, call it $\lambda$. Then, by definition 5.5, there exists $u\in V$ such that $u\neq0$ and $Tu=\lambda u$. That is, $u$ is an eigenvector of $T$ corresponding to $\lambda$. Put $v\equiv\frac{u}{\dnorm{u}}$ so that
$$ \dnorm{v}=\dnorm{\frac{u}{\dnorm{u}}}=\normw{\frac1{\dnorm{u}}}\dnorm{u}=\frac1{\dnorm{u}}\dnorm{u}=1 $$
and $v$ is an orthonormal basis of $V$. And $v$ is also an eigenvector of $T$:
$$ Tv=T\Prn{\frac{u}{\dnorm{u}}}=\frac1{\dnorm{u}}Tu=\frac1{\dnorm{u}}\lambda u=\lambda\frac1{\dnorm{u}}u=\lambda\Prn{\frac{u}{\dnorm{u}}}=\lambda v $$
Alternative proof without using (a): Since $\dim{V}=1$, then $V\neq\set{0}$. Hence there exists a nonzero vector $u\in V$. Put $v\equiv\frac{u}{\dnorm{u}}$ so that
$$ \dnorm{v}=\dnorm{\frac{u}{\dnorm{u}}}=\normw{\frac1{\dnorm{u}}}\dnorm{u}=\frac1{\dnorm{u}}\dnorm{u}=1 $$
Hence $v$ is an orthonormal basis of $V$. Hence every vector in $V$ is a scalar multiple of $v$. Since $Tv\in V$, then there exists a scalar $\lambda\in\wR$ such that $Tv=\lambda v$. That is, $v$ is a eigenvector of $T$ corresponding to the eigenvalue $\lambda$.
Analogy of Self-Adjoint to Real Numbers
p.209 “The adjoint on $\oper{V}$ plays a role similar to complex conjugation on $\wC$. A complex number $z$ is real if and only if $z=\cj{z}$; thus a self-adjoint operator ($T=\adjt{T}$) is analogous to a real number.”
My interpretation the procedure (for lack of a better word) of complex conjugation $\cj{z}$ is analogous to the procedure of adjoint $\adjt{T}$. That is, $z$ is real if and only if $z=\cj{z}$ and $T$ is self-adjoint if and only if $T=\adjt{T}$.
p.209 “We will see that this analogy is reflected in some important properties of self-adjoint operators, beginning with eigenvalues in the next result: 7.13 Every eigenvalue of a self-adjoint operator is real.”
My interpretation So I guess there is a correspondence $\wR$ ~ self-adjoint operators and a correspondence $\wC$ ~ non-self-adjoint operators.
p.210-211 “The next result provides another example of how self-adjoint operators behave like real numbers”, referring to 7.15.
My interpretation Self-adjoint operators behave like real numbers.
p.219 Pretty much whole page further draws out this analogy. Highlight: “Replacing the real number $x$ with a self-adjoint operator (recall the analogy between real numbers and self-adjoint operators), we are led to the result below.” That result is 7.26.
My interpretation Again $\wR$ ~ self-adjoint operators
p.223, exercise 7.B.2 Supose that $T$ is a self-adjoint operator on a finite-dimensional inner product space and that $2$ and $3$ are the only eigenvalues of $T$. Then $T^2-5T+6I=0$.
In my proof, I note that $b^2=(-5)^2=25\gt24=4\cdot6=4c$. So even though $T$ is self-adjoint, the result – that $T^2-5T+6I$ is not injective (by 3.16) and hence not invertible (by 3.69) – does not conflict with 7.26. This result further strengthens the analogy between the real numbers and self-adjoint operators because $2$ and $3$ are the only roots of the quadractic polynomial $x^2-5x+6=(x-2)(x-3)$.
My interpretation Again $\wR$ ~ self-adjoint operators
p.223, exercise 7.B.6 “A normal operator on a complex inner product space is self-adjoint if and only if all its eigenvalues are real.
Note: This exercise strengthens the analogy (for normal operators) between self-adjoint operators and real numbers.”
My interpretation Again $\wR$ ~ self-adjoint operators
p.224, exercise 7.B.10 Give an example of a real inner product space $V$ and $T\in\oper{V}$ and real number $b,c$ with $b^2\lt4c$ such that $T^2+bT+cI$ is not invertible.
Note: This exercise shows that the hypothesis that $T$ is self-adjoint is needed in 7.26, even for real vector spaces.
My interpretation Again $\wR$ ~ self-adjoint operators
p.225-226 The paragraph at the bottom of p.225 draws correspondences between the positive operators and nonnegative numbers among $\wC$. In particular, “$z$ is nonnegative if and only if it has a real square root, corresponding to condition (d): $T$ is positive if and only if $T$ has a self-adjoint square root”.
p.233 The first three paragraphs very nicely explain the full analogy.
General
Proposition W.7.G.12 If $T\in\oper{V}$ is normal, then so is $\adjt{T}$.
Proof We’re given that $\tta=\tat$. This gives the second equality:
$$ \adjt{T}(\adjt{T})^*=\tat=\tta=(\adjt{T})^*\adjt{T} $$
The first and last equalities follow from 7.6(c). $\blacksquare$
Proposition W.7.G.13 Suppose $T\in\oper{V}$, $U$ is a subspace of $V$, and $u$ is an eigenvector of $T\bar_U$. Then $u$ is an eigenvector of $T$.
Proof Let $\lambda$ be the eigenvalue of $T\bar_U$ corresponding to $u$. Then
$$ Tu=T\bar_Uu=\lambda u $$
$\blacksquare$
Definition 7.31 “If $V$ is a complex vector space, then the requirement that $T$ is self-adjoint can be dropped from the definition above (by 7.15).”
Explanation Suppose $V$ is a complex vector space and $T\in\oper{V}$ satisfies $\innprd{Tv}{v}\geq0$ for all $v\in V$. This implies that $\innprd{Tv}{v}$ is nonnegative and hence real for all $v\in V$. Then 7.15 implies that $T$ is self-adjoint. $\wes$
Proposition W.7.G.14 (Example 7.32) Let $U$ be a finite-dimensional subspace of the inner product space $V$. Then the orthogonal projection $P_U$ is a positive operator.
Proof Proposition 6.55(g) gives $P_U^2=P_U$. Then exercise 7.A.11 implies that $P$ is self-adjoint.
Let $v\in V$. Then $v$ can be uniquely decomposed as $v=u+w$ for some $u\in U$ and some $w\in U^\perp$ (by 6.47). Hence
$$ \innprd{P_Uv}{v} = \innprd{P_U(u+w)}{u+w} = \innprd{u}{u+w} = \innprd{u}{u}+\innprd{u}{w} = \innprd{u}{u} \geq 0 $$
$\blacksquare$
Proposition W.7.G.15 (Prop 7.35) Let $\lambda_1,\dots,\lambda_n$ be nonnegative and let $e_1,\dots,e_n$ be an orthonormal basis of $V$. Define $R\in\oper{V}$ by $Re_j\equiv\sqrt{\lambda_j}e_j$ for $j=1,\dots,n$. Then $R$ is a positive operator.
Proof Formula W.7.CoA.1 gives
$$ R^*e_j=\sum_{k=1}^n\innprd{e_j}{Re_k}e_k=\sum_{k=1}^n\innprd{e_j}{\sqrt{\lambda_k}e_k}e_k=\sum_{k=1}^n\cj{\sqrt{\lambda_k}}\innprd{e_j}{e_k}e_k=\sum_{k=1}^n\sqrt{\lambda_k}\mathbb{1}_{j=k}e_k=\sqrt{\lambda_j}e_j=Re_j $$
Hence $R^*=R$ and $R$ is self-adjoint. Let $v\in V$. Then $v=\sum_{k=1}^n\innprd{v}{e_k}e_k$ and
$$\align{ \innprd{Rv}{v} &= \innprdBgg{R\Prn{\sum_{j=1}^n\innprd{v}{e_j}e_j}}{\sum_{k=1}^n\innprd{v}{e_k}e_k} \\ &= \innprdBgg{\sum_{j=1}^n\innprd{v}{e_j}Re_j}{\sum_{k=1}^n\innprd{v}{e_k}e_k} \\ &= \sum_{j=1}^n\sum_{k=1}^n\innprdbg{\innprd{v}{e_j}Re_j}{\innprd{v}{e_k}e_k} \\ &= \sum_{j=1}^n\sum_{k=1}^n\innprd{v}{e_j}\cj{\innprd{v}{e_k}}\innprd{Re_j}{e_k} \\ &= \sum_{j=1}^n\sum_{k=1}^n\innprd{v}{e_j}\cj{\innprd{v}{e_k}}\innprd{\sqrt{\lambda_j}e_j}{e_k} \\ &= \sum_{j=1}^n\sum_{k=1}^n\innprd{v}{e_j}\cj{\innprd{v}{e_k}}\sqrt{\lambda_j}\innprd{e_j}{e_k} \\ &= \sum_{j=1}^n\sum_{k=1}^n\innprd{v}{e_j}\cj{\innprd{v}{e_k}}\sqrt{\lambda_j}\mathbb{1}_{j=k} \\ &= \sum_{j=1}^n\innprd{v}{e_j}\cj{\innprd{v}{e_j}}\sqrt{\lambda_j} \\ &= \sum_{j=1}^n\normb{\innprd{v}{e_j}}^2\sqrt{\lambda_j} \\ &\geq0 }$$
$\wes$
Proposition W.7.G.16 Let $T\wiov$. For any positive integer $k$, we have $(T^k)^*=(\adjt{T})^k$.
Proof Note that $(T^2)^*=(TT)^*=\tat^*=(\adjt{T})^2$. Our induction assumption is $(T^{k-1})^*=(\adjt{T})^{k-1}$. Then
$$ (T^k)^*=(TT^{k-1})^*=(T^{k-1})^*\adjt{T}=(\adjt{T})^{k-1}\adjt{T}=(\adjt{T})^k $$
The second equality follows from 7.6(e). $\wes$
Proposition W.7.G.17 Let $\wF=\wR$ and suppose $T\wiov$ has $\dim{V}$ distinct eigenvalues. Then the following are equivalent:
(a) $T$ is normal.
(b) $V$ has an orthonormal basis consisting of eigenvectors of $T$.
(c) $T$ has a diagonal matrix with respect to some orthonormal basis of $V$.
(d) $T$ is self-adjoint.
Proof (d) implies (a) since $T=\adjt{T}$ gives $\tta=TT=\tat$.
Now suppose (a) holds, so $T$ is normal. And put $n\equiv\dim{V}$. By Schur’s Theorem (6.38), there is an orthonormal basis $e_1,\dots,e_n$ of $V$ with respect to which $T$ has an upper-triangular matrix. Hence we can write
$$ \mtrxof{T,(e_1,\dots,e_n)}=\pmtrx{a_{1,1}&a_{1,2}&a_{1,3}&\dotsb&a_{1,n}\\0&a_{2,2}&a_{2,3}&\dotsb&a_{2,n}\\0&0&a_{3,3}&\dotsb&a_{3,n}\\0&0&0&\ddots&\vdots\\0&0&0&0&a_{n,n}} $$
By 5.32, the diagonal entries of $\mtrxof{T,(e_1,\dots,e_n)}$ are the eigenvalues of $T$. By hypothesis, we have $n=\dim{V}$ distinct eigenvalues of $T$. Hence $a_{j,j}=\lambda_j$ and
$$ \mtrxof{T,(e_1,\dots,e_n)}=\pmtrx{\lambda_1&a_{1,2}&a_{1,3}&\dotsb&a_{1,n}\\0&\lambda_2&a_{2,3}&\dotsb&a_{2,n}\\0&0&\lambda_3&\dotsb&a_{3,n}\\0&0&0&\ddots&\vdots\\0&0&0&0&\lambda_n} \tag{W.7.G.16} $$
We see that $Te_1=\lambda_1e_1$. And since $\wF=\wR$, then
$$ \adjt{T}e_1=\cj{\lambda_1}e_1+\sum_{j=2}^n\cj{a_{1,j}}e_j=\lambda_1e_1+\sum_{j=2}^na_{1,j}e_j $$
Hence, 6.25 gives
$$ \dnorm{\adjt{T}e_1}^2=\norm{\lambda_1}^2+\sum_{j=2}^n\norm{a_{1,j}}^2 $$
Since $T$ is normal, 7.20 gives
$$ \norm{\lambda_1}^2=\dnorm{Te_1}^2=\dnorm{\adjt{T}e_1}^2=\norm{\lambda_1}^2+\sum_{j=2}^n\norm{a_{1,j}}^2 $$
Hence
$$ 0=\sum_{j=2}^n\norm{a_{1,j}}^2 $$
and this implies that $0=a_{1,j}$ for $j=2,\dots,n$. In particular, $0=a_{1,2}$ and hence, from W.7.G.16, we see that $Te_2=\lambda_2e_2$. We also see that
$$ \adjt{T}e_2=\lambda_2e_2+\sum_{j=3}^na_{2,j}e_j $$
Hence, 6.25 gives
$$ \dnorm{\adjt{T}e_2}^2=\norm{\lambda_2}^2+\sum_{j=3}^n\norm{a_{2,j}}^2 $$
Since $T$ is normal, 7.20 gives
$$ \norm{\lambda_2}^2=\dnorm{Te_2}^2=\dnorm{\adjt{T}e_2}^2=\norm{\lambda_2}^2+\sum_{j=3}^n\norm{a_{2,j}}^2 $$
Hence
$$ 0=\sum_{j=3}^n\norm{a_{2,j}}^2 $$
and this implies that $0=a_{2,j}$ for $j=3,\dots,n$. In particular, $0=a_{2,3}$ and above we proved that $0=a_{1,3}$. Hence, from W.7.G.16, we see that $Te_3=\lambda_3e_3$… Continuing in this way, we find that $Te_j=\lambda_je_j$ for $j=1,\dots,n$. Hence $V$ has an orthonormal basis consisting of eigenvectors of $T$. Hence (a) implies (b).
That (b) implies (c) follows from 5.41(b)$\implies$5.41(a).
Now suppose (c) holds, so $\mtrxof{T}$ is diagonal with respect to some orthonormal basis of $V$. Then $\mtrxof{\adjt{T}}$ (with respect to the same basis) is obtained by taking the transpose ($\wF=\wR$) of $\mtrxof{T}$. Since diagonal matrices are symmetric and the transpose of a symmetric matrix is the original matrix, then $\mtrxof{\adjt{T}}=\mtrxof{T}$. $\wes$
Proposition W.7.G.18 Let $\wF=\wR$ and let $\lambda_1,\dots,\lambda_m$ denote the distinct eigenvalues of $T\wiov$. Suppose we have
$$ \dim{V}=\sum_{j=1}^m\dim{\eignsp{\lambda_j}{T}} $$
Then the following are equivalent:
(a) $T$ is normal.
(b) $V$ has an orthonormal basis consisting of eigenvectors of $T$.
(c) $T$ has a diagonal matrix with respect to some orthonormal basis of $V$.
(d) $T$ is self-adjoint.
Proof This proof is very similar to that of W.7.G.17. $\wes$
Example W.7.G.19, from Example 7.50, p.236 Define $T\in\oper{\wF^4}$ by
$$ T(z_1,z_2,z_3,z_4)=(0,3z_1,2z_2,-3z_4) $$
Find the singular values of $T$.
First let’s find $\adjt{T}$.
$$\align{ \innprdbg{(z_1,z_2,z_3,z_4)}{\adjt{T}(w_1,w_2,w_3,w_4)} &= \innprdbg{T(z_1,z_2,z_3,z_4)}{(w_1,w_2,w_3,w_4)} \\ &= \innprdbg{(0,3z_1,2z_2,-3z_4)}{(w_1,w_2,w_3,w_4)} \\ &= 3z_1w_2+2z_2w_3-3z_4w_4 \\ &= \innprdbg{(z_1,z_2,z_3,z_4)}{(3w_2,2w_3,0,-3w_4)} }$$
Hence
$$ \adjt{T}(w_1,w_2,w_3,w_4)=(3w_2,2w_3,0,-3w_4) \tag{W.7.G.18.1} $$
Note that
$$\align{ Te_1&=(0,3,0,0)=3e_2\\ Te_2&=(0,0,2,0)=2e_3\\ \tag{W.7.G.18.2} Te_3&=(0,0,0,0)=0e_3\\ Te_4&=(0,0,0,-3)=-3e_4 }$$
And we can check the computation W.7.G.18.1:
$$\align{ \adjt{T}(w_1,w_2,w_3,w_4) &= \sum_{k=1}^4\innprdbg{(w_1,w_2,w_3,w_4)}{Te_k}e_k\tag{by W.7.CoA.1} \\ &= \innprdbg{(w_1,w_2,w_3,w_4)}{(0,3,0,0)}e_1+\innprdbg{(w_1,w_2,w_3,w_4)}{(0,0,2,0)}e_2 \\ &+ \innprdbg{(w_1,w_2,w_3,w_4)}{(0,0,0,0)}e_3+\innprdbg{(w_1,w_2,w_3,w_4)}{(0,0,0,-3)}e_4 \\ &= 3w_2e_1+2w_3e_2+0e_3-3w_4e_4 \\ &= (3w_2,2w_3,0,-3w_4) }$$
Hence
$$ \tat(z_1,z_2,z_3,z_4)=\adjt{T}(0,3z_1,2z_2,-3z_4)=(3\cdot3z_1,2\cdot2z_2,0,-3\cdot-3z_4)=(9z_1,4z_2,0,9z_4) $$
and
$$\align{ \tat e_1&=(9,0,0,0)=9e_1\\ \tat e_2&=(0,4,0,0)=4e_2\\ \tag{W.7.G.18.3} \tat e_3&=(0,0,0,0)=0e_3\\ \tat e_4&=(0,0,0,9)=9e_4 }$$
Hence $\sqrt{\tat}(z_1,z_2,z_3,z_4)=(3z_1,2z_2,0,3z_4)$ since
$$\align{ \prn{\sqrt{\tat}}^2(z_1,z_2,z_3,z_4) &= \sqrt{\tat}\sqrt{\tat}(z_1,z_2,z_3,z_4) \\ &= \sqrt{\tat}(3z_1,2z_2,0,3z_4) \\ &= (3\cdot3z_1,2\cdot2z_2,0,3\cdot3z_4) \\ &= (9z_1,4z_2,0,9z_4) \\ &= \tat(z_1,z_2,z_3,z_4) }$$
Note that
$$\align{ \sqrt{\tat}e_1&=(3,0,0,0)=3e_1\\ \sqrt{\tat}e_2&=(0,2,0,0)=2e_2\\ \tag{W.7.G.18.4} \sqrt{\tat}e_3&=(0,0,0,0)=0e_3\\ \sqrt{\tat}e_4&=(0,0,0,3)=3e_4 }$$
Hence the eigenvalues of $\sqrt{\tat}$ are $3,2,0$. Note that
$$ \dim{\eignspb{3}{\sqrt{\tat}}}=2 \dq \dim{\eignspb{2}{\sqrt{\tat}}}=1 \dq \dim{\eignspb{0}{\sqrt{\tat}}}=1 $$
Hence the singular values of $T$ are $3,3,2,0$.
Note from W.7.G.18.2 that $-3$ and $0$ are the only eigenvalues of $T$. Hence, in this case, the collection of eigenvalues of $T$ did not pick up the number $2$ that appears in the definition (and hence the behavior) of $T$. But the collection of singular values does include $2$.
It’s helpful to view the matrices involved:
$$ \mtrxof{T}=\pmtrx{0&0&0&0\\3&0&0&0\\0&2&0&0\\0&0&0&-3} \dq \mtrxof{\tat}=\pmtrx{9&0&0&0\\0&4&0&0\\0&0&0&0\\0&0&0&9} \dq \mtrxofb{\sqrt{\tat}}=\pmtrx{3&0&0&0\\0&2&0&0\\0&0&0&0\\0&0&0&3} $$
Alternatively (7.52), we can avoid finding $\sqrt{\tat}$ by first computing the eigenvalues of $\tat$ and then taking their square roots. Hence, from W.7.G.18.3, we get that the singular values of $T$ are $\sqrt9,\sqrt4,\sqrt0,\sqrt9$ or $3,3,2,0$.
Clarification W.7.G.20 for 7.36, p.227 “Suppose $T\wiov$ is positive. Suppose $v\in V$ is an eigenvector of $T$ (by 7.27). Thus there exists $\lambda\geq0$ (by 7.35(b)) such that $Tv=\lambda v$. Let $R$ be a positive square root of $T$. We will prove that $Rv=\sqrt{\lambda}v$. This will imply that the behavior of $R$ on the eigenvectors of $T$ is uniquely determined. Because there is a basis of $V$ consisting of eigenvectors of $T$ (by the Spectral Theorem), this will imply that $R$ is uniquely determined.”
Axler goes on to show that $Rv=\sqrt{\lambda}v$. Since $\lambda$ and $v$ were chosen arbitrarily, then this is true for any eigenpair $\lambda,v$ of $T$. In particular
$$ Re_j=\sqrt{\lambda_j}e_j\qd\text{for }j=1,\dots,n \tag{W.7.G.20.1} $$
where $e_1,\dots,e_n$ is an orthonormal eigenbasis of $V$ corresponding to the nonnegative eigenvalues $\lambda_1,\dots,\lambda_n$ of $T$ (by the Spectral Theorem and 7.35(b)). But 3.5 gives that W.7.G.20.1 fully determines a unique operator $R\wiov$ whereas $R$ was chosen to be an arbitrary positive square root of $T$. Hence every positive square root of $T$ must satisfy the unique formula W.7.G.20.1.
$\wes$
Proposition W.7.G.21 Let $T\in\linmap{V}{W}$. Then $\tat\wiov$ is a positive operator.
Also, let $e_1,\dots,e_n$ denote an orthonormal basis of $V$. Then $e_1,\dots,e_n$ consists of eigenvectors of $\tat$ corresponding to the eigenvalues $\lambda_1,\dots,\lambda_n$ if and only if $e_1,\dots,e_n$ consists of eigenvectors of $\sqrttat$ corresponding to the eigenvalues $\sqrt{\lambda_1},\dots,\sqrt{\lambda_n}$. That is
$$ \tat e_j=\lambda_j e_j\qd\text{for }j=1,\dots,n \dq\iff\dq \sqrttat e_j=\sqrt{\lambda_j}e_j\qd\text{for }j=1,\dots,n $$
Proof Note that $\adjt{(\tat)}=\adjt{T}\adjt{(\adjt{T})}=\tat$. Hence $\tat$ is self-adjoint. Also note that $\innprd{\tat v}{v}=\innprd{Tv}{Tv}\geq 0$ for every $v\in V$. Hence $\tat$ is a positive operator.
Suppose $e_1,\dots,e_n$ consists of eigenvectors of $\tat$ corresponding to the eigenvalues $\lambda_1,\dots,\lambda_n$ (existence given by the Spectral Theorem). Then 7.35(b) implies that the eigenvalues $\lambda_1,\dots,\lambda_n$ of $\tat$ are all nonnegative. Now define $R\wiov$ by
$$ Re_j=\sqrt{\lambda_j}e_j\qd\text{for }j=1,\dots,n $$
Then $R$ is positive (by W.7.G.15) and, for $j=1,\dots,n$, we have
$$ R^2e_j=R\prn{\sqrt{\lambda_j}e_j}=\sqrt{\lambda_j}Re_j=\sqrt{\lambda_j}\sqrt{\lambda_j}e_j=\lambda_je_j=\tat e_j $$
Hence, by W.3.25, 7.36, and 7.44, we have $\sqrt{\tat}=R$.
Conversely, suppose $e_1,\dots,e_n$ consists of eigenvectors of $\sqrttat$ corresponding to the eigenvalues $\sqrt{\lambda_1},\dots,\sqrt{\lambda_n}$ (existence given by the Spectral Theorem). Then
$$ \tat e_j=\sqrttat\sqrttat e_j=\sqrttat\prn{\sqrt{\lambda_j}e_j}=\sqrt{\lambda_j}\sqrttat e_j=\sqrt{\lambda_j}\sqrt{\lambda_j}e_j=\lambda_je_j $$
for $j=1,\dots,n$. $\wes$
Proposition W.7.G.22 <–> 7.52 Let $n\equiv\dim{V}$ and let $T\wiov$. Then there are $n$ singular values of $T$ and they are the nonnegative square roots of the eigenvalues of $\tat$, with each eigenvalue $\lambda$ repeated $\dim{\eignsp{\lambda}{\tat}}$ times.
Proof From W.7.G.21, we see that
$$ \sqrt{\tat}e_j=\sqrt{\lambda_j}e_j\qd\text{for }j=1,\dots,n $$
where $e_1,\dots,e_n$ is an orthonormal eigenbasis of $V$ corresponding to the nonnegative eigenvalues $\lambda_1,\dots,\lambda_n$ of $\tat$. Hence $\sqrt{\tat}$ has $n$ nonnegative eigenvalues (counting multiples) and
$$ n=\sum_{k=1}^n\dim{\eignspb{\lambda_k}{\sqrt{\tat}}} $$
By definition (7.49), the singular values of $T$ are the eigenvalues of $\sqrt{\tat}$, with each eigenvalue $\lambda$ repeated $\dim{\eignsp{\lambda}{\sqrt{\tat}}}$ times. Define $s_j\equiv\sqrt{\lambda_j}$ for $j=1,\dots,n$ so that $s_1,\dots,s_n$ are the singular values of $T$. $\wes$
Proposition W.7.G.23 Suppose $T\wiov$ is invertible. Then $\adjtT$ is invertible and
$$ \inv{(\adjtT)}=\adjt{(\inv{T})} $$
Proof Since $T$ is invertible, then $\inv{T}$ is well defined. Then
$$ (\adjtT)\adjt{(\inv{T})}=\adjt{(\inv{T}T)}=\adjt{I}=I $$
and
$$ \adjt{(\inv{T})}(\adjtT)=\adjt{(T\inv{T})}=\adjt{I}=I $$
Hence $\adjtT$ is invertible. Since the inverse is unique (3.54), then $\inv{(\adjtT)}=\adjt{(\inv{T})}$. $\wes$
Proposition W.7.G.24 Let $T\wiov$. Then $T$ is invertible if and only if $\sqrt{\tat}$ is invertible.
Proof $\tat$ is positive since $\adjt{(\tat)}=\adjtT\adjt{(\adjtT)}=\tat$ and $\innprd{\tat v}{v}=\innprd{Tv}{Tv}\geq0$. Hence it has the positive square root $\sqrt{\tat}$ (by 7.36 and 7.44). By the definition of square root (7.33), we have
$$ \sqrt{\tat}\sqrt{\tat}=\prn{\sqrt{\tat}}^2=\tat \tag{W.7.G.24.1} $$
First suppose that $T$ is invertible. Then W.7.G.23 implies that $\adjtT$ is invertible. Then exercise 3.D.9 implies that $\tat$ is invertible. Hence, an application of exercise 3.D.9 to W.7.G.24.1 implies that $\sqrt{\tat}$ is invertible.
Conversely, suppose that $\sqrt{\tat}$ is invertible. Hence, applying exercise 3.D.9 to W.7.G.24.1, we get that $\tat$ is invertible. Hence, applying exercise 3.D.9 to $\tat$, we get that $T$ is invertible. $\wes$
Proposition W.7.G.25, 7.45, Polar Decomposition Let $T\wiov$ and define $S_1:\rangspb{\sqrttat}\mapsto\rangsp{T}$ by
$$ S_1\prn{\sqrttat v}\equiv Tv $$
Then
(a) $S_1$ is a well-defined function
(b) $S_1$ is a linear map
(c) $\dnorm{S_1u}=\dnorm{u}$ for all $u\in\rangspb{\sqrttat}$
(d) $S_1$ is injective
(e) $S_1$ is surjective
(f) $\innprd{S_1u_1}{S_1u_2}=\innprd{u_1}{u_2}$ for all $u_1,u_2\in\rangspb{\sqrttat}$
(g) If $e_1,\dots,e_m$ is an orthonormal list in $\rangspb{\sqrttat}$, then $S_1e_1,\dots,S_1e_m$ is an orthonormal list in $\rangsp{T}$
(h) If $e_1,\dots,e_m$ is an orthonormal basis of $\rangspb{\sqrttat}$, then $S_1e_1,\dots,S_1e_m$ is an orthonormal basis of $\rangsp{T}$
Proof of (a) In the proof of 7.45, Axler shows that $S_1$ is well-defined.
Proof of (b) Let $u_1,u_2\in\rangspb{\sqrttat}$. Then there exist $v_1,v_2\in V$ such that $u_1=\sqrttat v_1$ and $u_2=\sqrttat v_2$. Also let $\lambda\in\wF$. Then
$$\align{ S_1(\lambda u_1+u_2) &= S_1\prn{\lambda\sqrttat v_1+\sqrttat v_2} \\ &= S_1\prn{\sqrttat(\lambda v_1)+\sqrttat v_2} \\ &= S_1\prn{\sqrttat(\lambda v_1+v_2)} \\ &= T(\lambda v_1+v_2) \\ &= T(\lambda v_1)+Tv_2 \\ &= \lambda Tv_1+Tv_2 \\ &= \lambda S_1\prn{\sqrttat v_1}+S_1\prn{\sqrttat v_2} \\ &= \lambda S_1u_1+S_1u_2 }$$
Hence $S_1\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ is a linear map.
Proof of (c) Let $u\in\rangspb{\sqrttat}$. Then there exists $v\in V$ such that $u=\sqrttat v$. Hence
$$ \dnorm{S_1u}=\dnorm{S_1\prn{\sqrttat v}}=\dnorm{Tv}=\dnorm{\sqrttat v}=\dnorm{u} $$
The next-to-last equality follows from 7.46.
Proof of (d) Let $u\in\rangspb{\sqrttat}$ such that $S_1u=0$. Then
$$ 0=\dnorm{S_1u}=\dnorm{u} $$
By 6.10(a), $u=0$. Hence $\nullsp{S_1}=\set{0}$. Hence $S_1$ is injective (by 3.16).
Proof of (e) Let $t\in\rangsp{T}$. Then there exists $v\in V$ such that $Tv=t$. Hence
$$ t=Tv=S_1\prn{\sqrttat v}\in\rangsp{S_1} $$
Hence $\rangsp{T}\subset\rangsp{S_1}$.
Conversely, suppose $s\in\rangsp{S_1}$. Then there exists $u\in\rangspb{\sqrttat}$ such that $S_1u=s$. Then there exists $v\in V$ such that $u=\sqrttat v$. Hence
$$ s=S_1u=S_1\prn{\sqrttat v}=Tv\in\rangsp{T} $$
Hence $\rangsp{S_1}\subset\rangsp{T}$ and $\rangsp{S_1}=\rangsp{T}$. Hence $S_1$ is surjective.
Proof of (f) Let $u_1,u_2\in\rangspb{\sqrttat}$. If $V$ is a real inner product space, then
$$\align{ \innprd{S_1u_1}{S_1u_2} &= \frac{\dnorm{S_1u_1+S_1u_2}^2-\dnorm{S_1u_1-S_1u_2}^2}4\tag{by exercise 6.A.19} \\ &= \frac{\dnorm{S_1(u_1+u_2)}^2-\dnorm{S_1(u_1-u_2)}^2}4 \\ &= \frac{\dnorm{u_1+u_2}^2-\dnorm{u_1-u_2}^2}4\tag{because $\dnorm{S_1u}=\dnorm{u}$} \\ &= \innprd{u_1}{u_2}\tag{by exercise 6.A.19} }$$
If $V$ is a complex inner product space, then
$$\align{ \innprd{S_1u_1}{S_1u_2} &= \frac{\dnorm{S_1u_1+S_1u_2}^2-\dnorm{S_1u_1-S_1u_2}^2}4 \\ &+ \frac{\dnorm{S_1u_1+iS_1u_2}^2i-\dnorm{S_1u_1-iS_1u_2}^2i}4\tag{by exercise 6.A.20} \\ &= \frac{\dnorm{S_1(u_1+u_2)}^2-\dnorm{S_1(u_1-u_2)}^2}4 \\ &+ \frac{\dnorm{S_1(u_1+iu_2)}^2i-\dnorm{S_1(u_1-iu_2)}^2i}4 \\ &= \frac{\dnorm{u_1+u_2}^2-\dnorm{u_1-u_2}^2+\dnorm{u_1+iu_2}^2i-\dnorm{u_1-iu_2}^2i}4\tag{because $\dnorm{S_1u}=\dnorm{u}$} \\ &= \innprd{u_1}{u_2}\tag{by exercise 6.A.20} }$$
Proof of (g) Let $e_1,\dots,e_m$ be an orthonormal list in $\rangspb{\sqrttat}$. Since $\innprd{S_1e_j}{S_1e_k}=\innprd{e_j}{e_k}$ for any $j,k=1,\dots,m$, then $Se_1,\dots,Se_m$ is orthonormal in $\rangsp{T}$. $\wes$
Proof of (h) Since $S_1$ is injective, then $\nullsp{S_1}=\set{0}$. Hence $\dim{\nullsp{S_1}}=0$. And since $S_1$ is surjective, then $\rangsp{T}=\rangsp{S_1}$. Hence the Fundamental Theorem of Linear Maps (3.22) gives
$$ \dim{\rangspb{\sqrttat}}=\dim{\nullsp{S_1}}+\dim{\rangsp{S_1}}=0+\dim{\rangsp{T}}=\dim{\rangsp{T}} $$
Put $m\equiv\dim{\rangspb{\sqrttat}}=\dim{\rangsp{T}}$ and let $\cdts{e_1}{e_m}$ be an orthonormal basis of $\rangspb{\sqrttat}$. Then part (g) implies that $\cdts{S_1e_1}{S_1e_m}$ is an orthonormal list in $\rangsp{T}$. Hence 6.28 implies that $\cdts{S_1e_1}{S_1e_m}$ is an orthonormal basis of $\rangsp{T}$. $\wes$
Defintion W.7.G.26 Let $U\in\wC^{n,n}$. Then $U$ is unitary if $\adjt{U}U=U\adjt{U}=I$. Note that $U$ is square.
Defintion W.7.G.27 Let $U\in\wR^{n,n}$. Then $U$ is orthogonal if $U^tU=UU^t=I$. Note that $U$ is square.
Proposition W.7.G.28 Let $U\in\wF^{m,n}$. Then the columns of $U$ are orthonormal if and only if $\adjt{U}U=I_n$.
Note that $U$ need not be square for this to hold.
Proof Suppose the columns of $U$ are orthonormal. Then
$$ \innprd{u_{:,j}}{u_{:,k}}=\cases{1&j=k\\0&j\neq k} $$
and
$$\align{ \adjt{U}U &= \pmtrx{\cj{u_{1,1}}&\dotsb&\dotsb&\cj{u_{m,1}}\\\vdots&\ddots&\ddots&\vdots\\\cj{u_{1,n}}&\dotsb&\dotsb&\cj{u_{m,n}}}\pmtrx{u_{1,1}&\dotsb&u_{1,n}\\\vdots&\ddots&\vdots\\\vdots&\ddots&\vdots\\u_{m,1}&\dotsb&u_{m,n}} \\\\ &= \pmtrx{\sum_{j=1}^m\cj{u_{j,1}}u_{j,1}&\dotsb&\sum_{j=1}^m\cj{u_{j,1}}u_{j,n} \\ \vdots&\ddots&\vdots \\ \sum_{j=1}^m\cj{u_{j,n}}u_{j,1}&\dotsb&\sum_{j=1}^m\cj{u_{j,n}}u_{j,n} } \\\\ &= \pmtrx{\sum_{j=1}^mu_{j,1}\cj{u_{j,1}}&\dotsb&\sum_{j=1}^mu_{j,n}\cj{u_{j,1}} \\ \vdots&\ddots&\vdots \\ \sum_{j=1}^mu_{j,1}\cj{u_{j,n}}&\dotsb&\sum_{j=1}^mu_{j,n}\cj{u_{j,n}} } \\\\ &= \pmtrx{\innprd{u_{:,1}}{u_{:,1}}&\dotsb&\innprd{u_{:,n}}{u_{:,1}} \\ \vdots&\ddots&\vdots \\ \innprd{u_{:,1}}{u_{:,n}}&\dotsb&\innprd{u_{:,n}}{u_{:,n}} } \\\\ &= I_n }$$
Conversely, suppose that $\adjt{U}U=I_n$. Then
$$ I_n=\adjt{U}U =\pmtrx{\cj{u_{1,1}}&\dotsb&\dotsb&\cj{u_{m,1}}\\\vdots&\ddots&\ddots&\vdots\\\cj{u_{1,n}}&\dotsb&\dotsb&\cj{u_{m,n}}}\pmtrx{u_{1,1}&\dotsb&u_{1,n}\\\vdots&\ddots&\vdots\\\vdots&\ddots&\vdots\\u_{m,1}&\dotsb&u_{m,n}} = \pmtrx{\innprd{u_{:,1}}{u_{:,1}}&\dotsb&\innprd{u_{:,n}}{u_{:,1}} \\ \vdots&\ddots&\vdots \\ \innprd{u_{:,1}}{u_{:,n}}&\dotsb&\innprd{u_{:,n}}{u_{:,n}} } $$
Hence
$$ \innprd{u_{:,j}}{u_{:,k}}=\cases{1&j=k\\0&j\neq k} $$
Hence the columns of $U$ are orthonormal. $\wes$
Proposition W.7.G.29 Let $U\in\wF^{m,n}$. Then the rows of $U$ are orthonormal if and only if $U\adjt{U}=I$.
Note that $U$ need not be square for this to hold.
Proof Define $W\equiv\adjt{U}$. By W.7.G.28, the columns of $W$ are orthonormal if and only if $\adjt{W}W=I$. But the columns of $W$ are the conjugate rows of $U$. By W.6.G.12, the columns of $W$ are orthonormal if and only if the rows of $U$ are orthonormal. Also $\adjt{W}W=\adjt{(\adjt{U})}\adjt{U}=U\adjt{U}$. $\wes$
Proposition W.7.G.30 Let $U\in\wF^{n,n}$. Then the following are equivalent:
(a) $U$ is unitary (orthogonal)
(b) $\adjt{U}U=I$
(c) The columns of $U$ are orthonormal
(d) $U\adjt{U}=I$
(e) The rows of $U$ are orthonormal
(f) $\adjt{U}$ is unitary
(g) $U$ is invertible and $\inv{U}=\adjt{U}$
(h) $\innprd{Uw}{Uz}=\innprd{w}{z}$ for all $w,z\in\wF^n$
(i) $\dnorm{Uz}=\dnorm{z}$ for all $z\in\wF^n$
Proof Suppose (a) holds so that $U$ is unitary. By definition W.7.G.26, we have $\adjt{U}U=I$. Hence (a)$\implies$(b).
Suppose (b) holds so that $\adjt{U}U=I$. By W.3.34 (translated to matrices), we have $U\adjt{U}=I$. Hence (b)$\implies$(d).
Suppose (d) holds so that $U\adjt{U}=I$. By W.3.34 (translated to matrices), we have $\adjt{U}U=I$. Hence $\adjt{U}U=U\adjt{U}=I$. Hence $U$ is unitary. Hence (d)$\implies$(a).
So we have shown that (a)$\implies$(b)$\implies$(d)$\implies$(a). Hence we have (a)$\iff$(b)$\iff$(d). We already showed that (b)$\iff$(c) in W.7.G.28 and that (d)$\iff$(e) in W.7.G.29. Hence we have (a)$\iff$(b)$\iff$(c)$\iff$(d)$\iff$(e).
Suppose (a) holds so that $\adjt{U}U=U\adjt{U}=I$. Hence $\adjt{(\adjt{U})}\adjt{U}=U\adjt{U}=I$ and $\adjt{U}\adjt{(\adjt{U})}=\adjt{U}U=I$. Hence $\adjt{U}$ is unitary and (a)$\implies$(f).
Suppose (f) holds so that $\adjt{(\adjt{U})}\adjt{U}=\adjt{U}\adjt{(\adjt{U})}=I$. Hence $U\adjt{U}=\adjt{(\adjt{U})}\adjt{U}=I$ and $\adjt{U}U=\adjt{U}\adjt{(\adjt{U})}=I$. Hence $U$ is unitary and (f)$\implies$(a).
Suppose (a) holds so that $\adjt{U}U=U\adjt{U}=I$. Hence $\adjt{U}$ is the left and right inverse of $U$. Hence $U$ is invertible and $\inv{U}=\adjt{U}$. Hence (a)$\implies$(g).
Suppose (g) holds so that $U$ is invertible and $\inv{U}=\adjt{U}$. Hence $\adjt{U}$ is the left and right inverse of $U$. Hence $\adjt{U}U=U\adjt{U}=I$. Hence $U$ is unitary and (g)$\implies$(a).
Hence we have (a)$\iff$(b)$\iff$(c)$\iff$(d)$\iff$(e)$\iff$(f)$\iff$(g).
Suppose (b) holds so that $\adjt{U}U=I$. Then
$$ \innprd{Uw}{Uz}=\innprd{\adjt{U}Uw}{z}=\innprd{w}{z} $$
The first equality holds from W.7.G.34. Hence (b)$\implies$(h).
Suppose (h) holds so that $\innprd{Uw}{Uz}=\innprd{w}{z}$ for all $w,z\in\wF^n$. Let $e_1,\dots,e_n$ denote the standard basis on $\wF^n$. Then
$$ Ue_j = \pmtrx{u_{1,1}&\dotsb&u_{1,n}\\\vdots&\ddots&\vdots\\u_{n,1}&\dotsb&u_{n,n}}\pmtrx{0\\\vdots\\1\\\vdots\\0} = \pmtrx{u_{1,j}\\\vdots\\u_{n,j}} = u_{:,j} $$
Hence
$$ \innprd{u_{:,j}}{u_{:,k}}=\innprd{Ue_j}{Ue_k}=\innprd{e_j}{e_k}=\cases{1&j=k\\0&j\neq k} $$
Hence the columns of $U$ are orthonormal. Hence (h)$\implies$(c)$\implies$(b).
Hence we have (a)$\iff$(b)$\iff$(c)$\iff$(d)$\iff$(e)$\iff$(f)$\iff$(g)$\iff$(h).
Suppose (h) holds so that $\innprd{Uw}{Uz}=\innprd{w}{z}$ for all $w,z\in\wF^n$. Then
$$ \dnorm{Uz}=\sqrt{\innprd{Uz}{Uz}}=\sqrt{\innprd{z}{z}}=\dnorm{z} $$
Hence (h)$\implies$(i).
Suppose (i) holds so that $\dnorm{Uz}=\dnorm{z}$ for all $z\in\wF^n$. Then, just as we did in the proof of W.7.G.43(b), we can show that $\innprd{Uw}{Uz}=\innprd{w}{z}$ for all $w,z\in\wF^n$. Hence (i)$\implies$(h). $\wes$
Proposition W.7.G.31 Let $A\in\wF^{2,2}$ and let $w,z\in\wF^2$. Then $\innprd{Aw}{z}=\innprd{w}{\adjt{A}z}$.
Proof We have
$$\align{ Aw &= \pmtrx{a_{1,1}&a_{1,2}\\a_{2,1}&a_{2,2}}\pmtrx{w_1\\w_2} \\ &= \pmtrx{a_{1,1}w_1+a_{1,2}w_2\\a_{2,1}w_1+a_{2,2}w_2} \\ }$$
and
$$\align{ \innprd{Aw}{z} &= (Aw)_1\cj{z_1}+(Aw)_2\cj{z_2} \\ &= (a_{1,1}w_1+a_{1,2}w_2)\cj{z_1}+(a_{2,1}w_1+a_{2,2}w_2)\cj{z_2} }$$
and
$$\align{ \adjt{A}z &= \pmtrx{\cj{a_{1,1}}&\cj{a_{2,1}}\\\cj{a_{1,2}}&\cj{a_{2,2}}}\pmtrx{z_1\\z_2} \\ &= \pmtrx{\cj{a_{1,1}}z_1+\cj{a_{2,1}}z_2\\\cj{a_{1,2}}z_1+\cj{a_{2,2}}z_2} \\ }$$
and
$$\align{ \innprd{w}{\adjt{A}z} &= w_1\cj{(\adjt{Az})_1}+w_2\cj{(\adjt{Aw})_2} \\ &= w_1\cj{\prn{\cj{a_{1,1}}z_1+\cj{a_{2,1}}z_2}}+w_2\cj{\prn{\cj{a_{1,2}}z_1+\cj{a_{2,2}}z_2}} \\ &= w_1\Prn{\cj{\cj{a_{1,1}}}\;\cj{z_1}+\cj{\cj{a_{2,1}}}\;\cj{z_2}}+w_2\Prn{\cj{\cj{a_{1,2}}}\;\cj{z_1}+\cj{\cj{a_{2,2}}}\;\cj{z_2}} \\ &= w_1\prn{a_{1,1}\cj{z_1}+a_{2,1}\cj{z_2}}+w_2\prn{a_{1,2}\cj{z_1}+a_{2,2}\cj{z_2}} \\ &= a_{1,1}w_1\cj{z_1}+a_{2,1}w_1\cj{z_2}+a_{1,2}w_2\cj{z_1}+a_{2,2}w_2\cj{z_2} \\ &= a_{1,1}w_1\cj{z_1}+a_{1,2}w_2\cj{z_1}+a_{2,1}w_1\cj{z_2}+a_{2,2}w_2\cj{z_2} \\ &= (a_{1,1}w_1+a_{1,2}w_2)\cj{z_1}+(a_{2,1}w_1+a_{2,2}w_2)\cj{z_2} \\ &= \innprd{Aw}{z} }$$
$\wes$
Proposition W.7.G.33 Let $A\in\wF^{2,2}$ and let $w,z\in\wF^2$. Then $\innprd{\adjt{A}Aw}{z}=\innprd{Aw}{Az}$.
Proof We have
$$\align{ \adjt{A}Aw &= \pmtrx{\cj{a_{1,1}}&\cj{a_{2,1}}\\\cj{a_{1,2}}&\cj{a_{2,2}}}\pmtrx{a_{1,1}&a_{1,2}\\a_{2,1}&a_{2,2}}\pmtrx{w_1\\w_2} \\ &= \pmtrx{\cj{a_{1,1}}a_{1,1}+\cj{a_{2,1}}a_{2,1}&\cj{a_{1,1}}a_{1,2}+\cj{a_{2,1}}a_{2,2}\\\cj{a_{1,2}}a_{1,1}+\cj{a_{2,2}}a_{2,1}&\cj{a_{1,2}}a_{1,2}+\cj{a_{2,2}}a_{2,2}}\pmtrx{w_1\\w_2} \\ &= \pmtrx{w_1\prn{\cj{a_{1,1}}a_{1,1}+\cj{a_{2,1}}a_{2,1}}+w_2\prn{\cj{a_{1,1}}a_{1,2}+\cj{a_{2,1}}a_{2,2}}\\w_1\prn{\cj{a_{1,2}}a_{1,1}+\cj{a_{2,2}}a_{2,1}}+w_2\prn{\cj{a_{1,2}}a_{1,2}+\cj{a_{2,2}}a_{2,2}}} \\ }$$
and
$$\align{ \innprd{\adjt{A}Aw}{z} &= \sum_{i=1}^n(\adjt{A}Aw)_i\cj{z_i} \\ &= \Prn{w_1\prn{\cj{a_{1,1}}a_{1,1}+\cj{a_{2,1}}a_{2,1}}+w_2\prn{\cj{a_{1,1}}a_{1,2}+\cj{a_{2,1}}a_{2,2}}}\cj{z_1} \\ &+ \Prn{w_1\prn{\cj{a_{1,2}}a_{1,1}+\cj{a_{2,2}}a_{2,1}}+w_2\prn{\cj{a_{1,2}}a_{1,2}+\cj{a_{2,2}}a_{2,2}}}\cj{z_2} \\ }$$
and
$$\align{ Az &= \pmtrx{a_{1,1}&a_{1,2}\\a_{2,1}&a_{2,2}}\pmtrx{z_1\\z_2} \\ &= \pmtrx{a_{1,1}z_1+a_{1,2}z_2\\a_{2,1}z_1+a_{2,2}z_2} \\ }$$
and
$$\align{ \innprd{Aw}{Az} &= \sum_{i=1}^n(Aw)_i\cj{(Az)_i} \\ &= \prn{a_{1,1}w_1+a_{1,2}w_2}\cj{\prn{a_{1,1}z_1+a_{1,2}z_2}}+\prn{a_{2,1}w_1+a_{2,2}w_2}\cj{\prn{a_{2,1}z_1+a_{2,2}z_2}} \\\\ &= \prn{a_{1,1}w_1+a_{1,2}w_2}\prn{\cj{a_{1,1}}\;\cj{z_1}+\cj{a_{1,2}}\;\cj{z_2}}+\prn{a_{2,1}w_1+a_{2,2}w_2}\prn{\cj{a_{2,1}}\;\cj{z_1}+\cj{a_{2,2}}\;\cj{z_2}} \\\\ &= a_{1,1}w_1\cj{a_{1,1}}\;\cj{z_1}+a_{1,1}w_1\cj{a_{1,2}}\;\cj{z_2}+a_{1,2}w_2\cj{a_{1,1}}\;\cj{z_1}+a_{1,2}w_2\cj{a_{1,2}}\;\cj{z_2} \\ &+ a_{2,1}w_1\cj{a_{2,1}}\;\cj{z_1}+a_{2,1}w_1\cj{a_{2,2}}\;\cj{z_2}+a_{2,2}w_2\cj{a_{2,1}}\;\cj{z_1}+a_{2,2}w_2\cj{a_{2,2}}\;\cj{z_2} \\\\ &= a_{1,1}w_1\cj{a_{1,1}}\;\cj{z_1}+a_{2,1}w_1\cj{a_{2,1}}\;\cj{z_1}+a_{1,2}w_2\cj{a_{1,1}}\;\cj{z_1}+a_{2,2}w_2\cj{a_{2,1}}\;\cj{z_1} \\ &+ a_{1,1}w_1\cj{a_{1,2}}\;\cj{z_2}+a_{2,1}w_1\cj{a_{2,2}}\;\cj{z_2}+a_{1,2}w_2\cj{a_{1,2}}\;\cj{z_2}+a_{2,2}w_2\cj{a_{2,2}}\;\cj{z_2} \\\\ &= \prn{a_{1,1}w_1\cj{a_{1,1}}+a_{2,1}w_1\cj{a_{2,1}}+a_{1,2}w_2\cj{a_{1,1}}+a_{2,2}w_2\cj{a_{2,1}}}\cj{z_1} \\ &+ \prn{a_{1,1}w_1\cj{a_{1,2}}+a_{2,1}w_1\cj{a_{2,2}}+a_{1,2}w_2\cj{a_{1,2}}+a_{2,2}w_2\cj{a_{2,2}}}\cj{z_2} \\\\ &= \Prn{w_1\prn{\cj{a_{1,1}}a_{1,1}+\cj{a_{2,1}}a_{2,1}}+w_2\prn{\cj{a_{1,1}}a_{1,2}+\cj{a_{2,1}}a_{2,2}}}\cj{z_1} \\ &+ \Prn{w_1\prn{\cj{a_{1,2}}a_{1,1}+\cj{a_{2,2}}a_{2,1}}+w_2\prn{\cj{a_{1,2}}a_{1,2}+\cj{a_{2,2}}a_{2,2}}}\cj{z_2} \\ &= \innprd{\adjt{A}Aw}{z} }$$
$\wes$
Proposition W.7.G.34 Let $A\in\wF^{n,n}$ and let $w,z\in\wF^n$. Then $\innprd{\adjt{A}Aw}{z}=\innprd{Aw}{Az}$.
Proof We have
$$\align{ \adjt{A}Aw &= \pmtrx{\cj{a_{1,1}}&\dotsb&\cj{a_{n,1}}\\\vdots&\ddots&\vdots\\\cj{a_{1,n}}&\dotsb&\cj{a_{n,n}}}\pmtrx{a_{1,1}&\dotsb&a_{1,n}\\\vdots&\ddots&\vdots\\a_{n,1}&\dotsb&a_{n,n}}\pmtrx{w_1\\\vdots\\w_n} \\\\ &= \pmtrx{\sum_{j=1}^n\cj{a_{j,1}}a_{j,1}&\dotsb&\sum_{j=1}^n\cj{a_{j,1}}a_{j,n}\\\vdots&\ddots&\vdots\\\sum_{j=1}^n\cj{a_{j,n}}a_{j,1}&\dotsb&\sum_{j=1}^n\cj{a_{j,n}}a_{j,n}}\pmtrx{w_1\\\vdots\\w_n} \\\\ &= \pmtrx{w_1\sum_{j=1}^n\cj{a_{j,1}}a_{j,1}+\dotsb+w_n\sum_{j=1}^n\cj{a_{j,1}}a_{j,n}\\\vdots\\w_1\sum_{j=1}^n\cj{a_{j,n}}a_{j,1}+\dotsb+w_n\sum_{j=1}^n\cj{a_{j,n}}a_{j,n}} \\\\ &= \pmtrx{\sum_{k=1}^nw_k\sum_{j=1}^n\cj{a_{j,1}}a_{j,k}\\\vdots\\\sum_{k=1}^nw_k\sum_{j=1}^n\cj{a_{j,n}}a_{j,k}} }$$
Hence
$$\align{ \innprd{\adjt{A}Aw}{z} &= \sum_{i=1}^n(\adjt{A}Aw)_i\cj{z_i} \\ &= \sum_{i=1}^n\Prn{\sum_{k=1}^nw_k\sum_{j=1}^n\cj{a_{j,i}}a_{j,k}}\cj{z_i} \\ &= \sum_{i=1}^n\sum_{k=1}^nw_k\sum_{j=1}^n\cj{a_{j,i}}a_{j,k}\cj{z_i} \\ }$$
On the other hand
$$\align{ Az &= \pmtrx{a_{1,1}&\dotsb&a_{1,n}\\\vdots&\ddots&\vdots\\a_{n,1}&\dotsb&a_{n,n}}\pmtrx{z_1\\\vdots\\z_n} \\\\ &= \pmtrx{\sum_{k=1}^nz_ka_{1,k}\\\vdots\\\sum_{k=1}^nz_ka_{n,k}} }$$
Hence
$$\align{ \innprd{Aw}{Az} &= \sum_{i=1}^n(Aw)_i\cj{(Az)_i} \\ &= \sum_{i=1}^n\Prn{\sum_{k=1}^nw_ka_{i,k}}\cj{\Prn{\sum_{j=1}^nz_ja_{i,j}}} \\ &= \sum_{i=1}^n\Prn{\sum_{k=1}^nw_ka_{i,k}}\Prn{\sum_{j=1}^n\cj{z_ja_{i,j}}} \\ &= \sum_{i=1}^n\Prn{\sum_{k=1}^nw_ka_{i,k}}\Prn{\sum_{j=1}^n\cj{z_j}\;\cj{a_{i,j}}} \\ &= \sum_{i=1}^n\sum_{k=1}^nw_ka_{i,k}\sum_{j=1}^n\cj{z_j}\;\cj{a_{i,j}} \\ &= \sum_{i=1}^n\sum_{k=1}^n\sum_{j=1}^nw_k\cj{a_{i,j}}a_{i,k}\;\cj{z_j} \\ &= \sum_{j=1}^n\sum_{k=1}^n\sum_{i=1}^nw_k\cj{a_{i,j}}a_{i,k}\;\cj{z_j} \\ &= \sum_{j=1}^n\sum_{k=1}^nw_k\sum_{i=1}^n\cj{a_{i,j}}a_{i,k}\;\cj{z_j} \\ &= \innprd{\adjt{A}Aw}{z} }$$
$\wes$
Proposition W.7.G.35 <–> 7.10 Let $T\in\linmap{V}{W}$, let $e\equiv e_1,\dots,e_n$ be an orthonormal basis of $V$, and let $f\equiv f_1,\dots,f_m$ be an orthonormal basis of $W$. Then
$$ \adjt{\mtrxof{T,e,f}}=\mtrxof{\adjtT,f,e} $$
Proof Note that 6.30 gives
$$ Te_k=\sum_{j=1}^m\innprd{Te_k}{f_j}_Wf_j $$
$$ \adjt{T}f_k=\sum_{j=1}^n\innprd{\adjt{T}f_k}{e_j}_Ve_j $$
Hence
$$\align{ \adjt{\mtrxof{T,e,f}} &= \adjt{\pmtrx{\mtrxof{T,e,f}_{:,1}&\dotsb&\mtrxof{T,e,f}_{:,n}}} \\\\ &= \adjt{\pmtrx{\mtrxof{Te_1,f}&\dotsb&\mtrxof{Te_n,f}}} \\\\ &= \adjt{\pmtrx{\innprd{Te_1}{f_1}_W&\dotsb&\innprd{Te_n}{f_1}_W\\\vdots&\ddots&\vdots\\\innprd{Te_1}{f_m}_W&\dotsb&\innprd{Te_n}{f_m}_W}} \\\\ &= \pmtrx{\cj{\innprd{Te_1}{f_1}_W}&\dotsb&\cj{\innprd{Te_1}{f_m}_W}\\\vdots&\ddots&\vdots\\\cj{\innprd{Te_n}{f_1}_W}&\dotsb&\cj{\innprd{Te_n}{f_m}_W}} \\\\ &= \pmtrx{\innprd{f_1}{Te_1}_W&\dotsb&\innprd{f_m}{Te_1}_W\\\vdots&\ddots&\vdots\\\innprd{f_1}{Te_n}_W&\dotsb&\innprd{f_m}{Te_n}_W} \\\\ &= \pmtrx{\innprd{\adjt{T}f_1}{e_1}_V&\dotsb&\innprd{\adjt{T}f_m}{e_1}_V\\\vdots&\ddots&\vdots\\\innprd{\adjt{T}f_1}{e_n}_V&\dotsb&\innprd{\adjt{T}f_m}{e_n}_V} \\\\ &= \pmtrx{\mtrxof{\adjt{T}f_1,e}&\dotsb&\mtrxof{\adjt{T}f_m,e}} \\\\ &= \pmtrx{\mtrxof{\adjt{T},f,e}_{:,1}&\dotsb&\mtrxof{\adjt{T},f,e}_{:,m}} \\\\ &= \mtrxof{\adjtT,f,e} }$$
$\wes$
Proposition W.7.G.36 Let $S\wiov$. Then $S$ is an isometry if and only if, for any orthonormal bases $\epsilon$ and $\varphi$ of $V$, the columns and rows of $\mtrxof{S,\epsilon,\varphi}$ are orthonormal with respect the usual dot product $\innprd{\cdot}{\cdot}_{\wF^n}$ of $\wF^n$ (see W.6.G.9).
Proof Let $e\equiv e_1,\dots,e_n$ and $f\equiv f_1,\dots,f_n$ be orthonormal bases of $V$.
First suppose that $S$ is an isometry. By 7.42(c), $Se_1,\dots,Se_n$ is orthonormal and hence is an orthonormal basis of $V$ (by 6.28). Also note that 6.30 gives
$$ Se_k=\sum_{i=1}^n\innprd{Se_k}{f_i}_Vf_i $$
Then
$$\align{ \mtrxof{S,e,f} &= \pmtrx{\mtrxof{S,e,f}_{:,1}&\dotsb&\mtrxof{S,e,f}_{:,n}} \\ &= \pmtrx{\mtrxof{Se_1,f}&\dotsb&\mtrxof{Se_n,f}} \\ &= \pmtrx{\innprd{Se_1}{f_1}_V&\dotsb&\innprd{Se_n}{f_1}_V\\\vdots&\ddots&\vdots\\\innprd{Se_1}{f_n}_V&\dotsb&\innprd{Se_n}{f_n}_V} \\ }$$
and
$$\align{ \innprdbg{\mtrxof{S,e,f}_{:,j}}{\mtrxof{S,e,f}_{:,k}}_{\wF^n} &= \sum_{i=1}^n\innprd{Se_j}{f_i}_V\cj{\innprd{Se_k}{f_i}_V}\tag{by W.6.G.9} \\ &= \sum_{i=1}^n\innprdbg{Se_j}{\innprd{Se_k}{f_i}_Vf_i}_V \\ &= \innprdBgg{Se_j}{\sum_{i=1}^n\innprd{Se_k}{f_i}_Vf_i}_V \\ &= \innprd{Se_j}{Se_k}_V\tag{by 6.30} \\ &= \cases{1&j=k\\0&j\neq k} }$$
The last equality holds because $Se_1,\dots,Se_n$ is orthonormal. Hence the columns of $\mtrxof{S,e,f}$ are orthonormal with respect to $\innprd{\cdot}{\cdot}_{\wF^n}$. Similarly
$$\align{ \innprdbg{\mtrxof{S,e,f}_{j,:}}{\mtrxof{S,e,f}_{k,:}}_{\wF^n} &= \sum_{i=1}^n\innprd{Se_i}{f_j}_V\cj{\innprd{Se_i}{f_k}_V}\tag{by W.6.G.9} \\ &= \sum_{i=1}^n\cj{\innprd{f_j}{Se_i}_V}\innprd{f_k}{Se_i}_V \\ &= \sum_{i=1}^n\innprdbg{f_k}{\innprd{f_j}{Se_i}_VSe_i}_V \\ &= \innprdBgg{f_k}{\sum_{i=1}^n\innprd{f_j}{Se_i}_VSe_i}_V \\ &= \innprd{f_k}{f_j}_V\tag{by 6.30} \\ &= \cases{1&j=k\\0&j\neq k} }$$
The last equality holds because $f_1,\dots,f_n$ is orthonormal. Hence the rows of $\mtrxof{S,e,f}$ are orthonormal with respect to $\innprd{\cdot}{\cdot}_{\wF^n}$.
Conversely, suppose that, for any orthonormal bases $\epsilon$ and $\varphi$ of $V$, the columns and rows of $\mtrxof{S,\epsilon,\varphi}$ are orthonormal with respect $\innprddt_{\wF^n}$. Then
$$\align{ \mtrxof{\adjt{S}S,e,e} &= \mtrxof{\adjt{S},e,e}\mtrxof{S,e,e}\tag{by W.3.27} \\ &= \adjt{\mtrxof{S,e,e}}\mtrxof{S,e,e}\tag{by W.7.G.35} \\ &= I_n\tag{by W.7.G.28} \\ &= \mtrxof{I,e,e} }$$
Since $\mtrxof{\cdot,e,e}$ is an isomorphism (by 3.60), then it’s injective (by 3.56). Hence $\adjt{S}S=I$ and $S$ is an isometry (by 7.42(e)). $\wes$
Proposition W.7.G.37 Let $S\wiov$. Then $S$ is an isometry if and only if, for every orthonormal basis $e$ of $V$, $\mtrxof{S,e}$ is orthogonal ($\wF=\wR$) or unitary ($\wF=\wC$) with respect to the usual inner product $\innprddt_{\wF^n}$ (see W.6.G.9).
Proof Let $e$ be an orthonormal basis of $V$.
First suppose $S$ is an isometry. By W.7.G.36, the columns of $\mtrxof{S,e}$ are orthonormal. By W.7.G.30, $\mtrxof{S,e}$ is orthogonal (unitary).
Conversely, suppose $\mtrxof{S,e}$ is orthogonal (unitary). By W.7.G.30, the columns of $\mtrxof{S,e}$ are orthonormal. Since $e$ was chosen arbitrarily, by W.7.G.36, $S$ is an isometry.
NOTE My intention with this proposition is to connect an isometry on an abstract inner product space with its orthogonal or unitary matrix. It seems that the latter can be used to build intuition and might be easier to work with preliminarily or computationally.
NOTE 2 At the top-right of p.229, Axler notes “An isometry on a real inner product space is often called an orthogonal operator. An isometry on a complex inner product space is often called a unitary operator. We use the term isometry so that our results can apply to both real and complex inner product spaces.”
$\wes$
Proposition W.7.G.38 Let $S\wiov$. Then $S$ is an isometry if and only if $\mtrxof{\adjt{S}S,e}=\mtrxof{I,e}$ for every orthonormal basis $e$ of $V$.
Proof Let $f\equiv f_1,\dots,f_n$ be an orthonormal basis of $V$.
First suppose $S$ is an isometry. Then $\adjt{S}S=I$ (by 7.42(e)). In particular, $\adjt{S}Sf_j=If_j$ for $j=1,\dots,n$. Hence $\mtrxof{\adjt{S}S,f}=\mtrxof{I,f}$. Since $f$ was chosen arbitrarily, then this is true for every orthonormal basis of $V$.
Conversely, suppose $\mtrxof{\adjt{S}S,e}=\mtrxof{I,e}$ for every orthonormal basis $e$ of $V$. Hence $\mtrxof{\adjt{S}S,f}=\mtrxof{I,f}$. Since $\mtrxof{\cdot,f}$ is an isomorphism (by 3.60), then it’s injective (by 3.56). Hence $\adjt{S}S=I$ and $S$ is an isometry (by 7.42(e)). $\wes$
Proposition W.7.G.39 An isometry is an isomorphism if and only if the codomain is the same size as the domain.
Proof Let $T\in\linmap{V}{W}$ such that $\dnorm{Tv}_W=\dnorm{v}_V$ for all $v\in V$.
First suppose $\dim{V}=\dim{W}$. Let $u\in V$ such that $Tu=0$. Then
$$ 0=\dnorm{Tu}_W=\dnorm{u}_V $$
By 6.10(a), $u=0$. Hence $\nullsp{T}=\set{0}$. Hence $T$ is injective.
Let $e_1,\dots,e_n$ be an orthonormal basis of $V$. Then $Te_1,\dots,Te_n$ is an orthonormal list (by 7.42(d)) in $W$ of the right length. Hence $Te_1,\dots,Te_n$ is an orthonormal basis of $W$ (by 6.28). Hence $T$ is surjective (by W.3.11).
Since $T$ is surjective and injective, then it’s an isomorphism (by 3.56).
Conversely, suppose $T$ is an isomorphism. Then $T$ is injective and surjective (by 3.56). Since $T$ is surjective, then $\dim{W}\leq\dim{V}$ (by 3.24). Since $T$ is injective, then $\dim{W}\geq\dim{V}$. Hence $\dim{W}=\dim{V}$. $\wes$
Proposition W.7.G.40 The product of two matrices, both with orthonormal columns, has orthonormal columns.
Proof Suppose $A\in\wFmn$ and $B\in\wF^{n\times p}$ both have orthonormal columns. By W.7.G.28, we have
$$ \AaA=I \dq\dq \adjt{B}B=I $$
Hence
$$ \adjt{(AB)}AB=\adjt{B}\adjt{A}AB=\adjt{B}IB=\adjt{B}B=I $$
By W.7.G.28, $AB$ has orthonormal columns. $\wes$
Proposition W.7.G.41 Let $T\wiov$ and suppose $V$ has an orthogonal basis $f_1,\dots,f_n$ consisting of eigenvectors of $T$. Then the list $Tf_1,\dots,Tf_n$ is orthogonal.
Proof For $j=1,\dots,n$, we have
$$ Tf_j=\lambda_jf_j $$
Hence, for $j\neq k$, we have
$$ \innprd{Tf_j}{Tf_k}=\innprd{\lambda_jf_j}{\lambda_kf_k}=\lambda_j\cj{\lambda_k}\innprd{f_j}{f_k}=0 $$
$\wes$
Example W.7.G.42 Define $T\in\linmap{\wR^3}{\wR^2}$ by
$$ T(1,0,0)\equiv(1,0) \dq T(0,1,0)\equiv(0,1) \dq T(0,0,1)\equiv(-1,0) $$
Then $T$ is not an isometry:
$$\align{ T(x,y,z) &= T\prn{(x,0,0)+(0,y,0)+(0,0,z)} \\ &= T\prn{x(1,0,0)+y(0,1,0)+z(0,0,1)} \\ &= xT(1,0,0)+yT(0,1,0)+zT(0,0,1) \\ &= x(1,0)+y(0,1)+z(-1,0) \\ &= (x,0)+(0,y)+(-z,0) \\ &= (x-z,y) }$$
Hence
$$\align{ \dnorm{T(x,y,z)} &= \dnorm{(x-z,y)} \\ &= \sqrt{(x-z)^2+y^2} \\ &= \sqrt{x^2-2xz+z^2+y^2} \\ }$$
whereas
$$ \dnorm{(x,y,z)}=\sqrt{x^2+y^2+z^2} $$
$\wes$
Proposition W.7.G.43<–>7.42 Let $S\in\linmap{V}{W}$ with $\dim{V}=\dim{W}$. Then the following are equivalent:
(a) $S$ is an isometry
(b) $\innprd{Su}{Sv}_W=\innprd{u}{v}_V$ for all $u,v\in V$
(c) $Se_1,\dots,Se_m$ is orthonormal for every orthonormal list $e_1,\dots,e_m$ in $V$
(d) there exists an orthonormal basis $e_1,\dots,e_n$ of $V$ such that $Se_1,\dots,Se_n$ is orthonormal
(e) $\adjt{S}S=I_V$
(f) $S\adjt{S}=I_W$
(g) $\adjt{S}$ is an isometry
(h) $S$ is invertible and $\inv{S}=\adjt{S}$
Proof (a)$\implies$(b) Let $u,v\in V$. If $V$ is a real inner product space, then
$$\align{ \innprd{Su}{Sv}_W &= \frac{\dnorm{Su+Sv}_W^2-\dnorm{Su-Sv}_W^2}4\tag{by exercise 6.A.19} \\ &= \frac{\dnorm{S(u+v)}_W^2-\dnorm{S(u-v)}_W^2}4 \\ &= \frac{\dnorm{u+v}_V^2-\dnorm{u-v}_V^2}4\tag{because $S$ isometric} \\ &= \innprd{u}{v}_V\tag{by exercise 6.A.19} }$$
If $V$ is a complex inner product space, then
$$\align{ \innprd{Su}{Sv}_W &= \frac{\dnorm{Su+Sv}_W^2-\dnorm{Su-Sv}_W^2}4 \\ &+ \frac{\dnorm{Su+iSv}_W^2i-\dnorm{Su-iSv}_W^2i}4\tag{by exercise 6.A.20} \\ &= \frac{\dnorm{S(u+v)}_W^2-\dnorm{S(u-v)}_W^2}4 \\ &+ \frac{\dnorm{S(u+iv)}_W^2i-\dnorm{S(u-iv)}_W^2i}4 \\ &= \frac{\dnorm{u+v}_V^2-\dnorm{u-v}_V^2+\dnorm{u+iv}_V^2i-\dnorm{u-iv}_V^2i}4\tag{because $S$ isometric} \\ &= \innprd{u}{v}_V\tag{by exercise 6.A.20} }$$
Proof (b)$\implies$(c) Let $e_1,\dots,e_m$ be an orthonormal list in $V$. Then
$$ \innprd{Se_j}{Se_k}=\innprd{e_j}{e_k}=\cases{1&j=k\\0&j\neq k} $$
Proof (c)$\implies$(d) This is clear.
Proof (d)$\implies$(e) Let $e_1,\dots,e_n$ denote an orthonormal basis of $V$. Then, for any $j,k=1,\dots,n$, we have
$$ \innprd{\adjt{S}Se_j}{e_k}=\innprd{Se_j}{Se_k}=\innprd{e_j}{e_k} $$
and W.6.G.19 implies that $\adjt{S}S=I_V$.
Proof (e)$\implies$(f) This follows from W.3.34.
Proof (f)$\implies$(g) For any $w\in W$, we have
$$ \dnorm{\adjt{S}w}_V^2=\innprd{\adjt{S}w}{\adjt{S}w}_V=\innprd{S\adjt{S}w}{w}_W=\innprd{I_Ww}{w}_W=\innprd{w}{w}_W=\dnorm{w}_W^2 $$
Proof (g)$\implies$(h) We know that (a)$\implies$(e). Hence, since $\adjt{S}$ is an isometry, then
$$ I_W=\adjt{(\adjt{S})}\adjt{S}=S\adjt{S} $$
And we know that (a)$\implies$(f). Hence, since $\adjt{S}$ is an isometry, then
$$ I_V=\adjt{S}\adjt{(\adjt{S})}=\adjt{S}S $$
Hence, by definition 3.53, $S$ is invertible and $\inv{S}=\adjt{S}$.
Proof (h)$\implies$(a) Since $S$ is invertible and $\inv{S}=\adjt{S}$, then $\adjt{S}S=I_V$. Hence, for any $v\in V$, we have
$$ \dnorm{Sv}_W^2=\innprd{Sv}{Sv}_W=\innprd{\adjt{S}Sv}{v}_V=\innprd{I_Vv}{v}_V=\innprd{v}{v}_V=\dnorm{v}_V^2 $$
$\wes$
Proposition W.7.G.44<–>W.7.G.36 Suppose $V$ and $W$ are finite-dimensional with $\dim{V}=\dim{W}$. Then $S\in\linmap{V}{W}$ is an isometry if and only if, for any orthonormal bases $\epsilon$ of $V$ and $\varphi$ of $W$, the columns and rows of $\mtrxof{S,\epsilon,\varphi}$ are orthonormal with respect the usual dot product $\innprd{\cdot}{\cdot}_{\wF^n}$ of $\wF^n$ (see W.6.G.9).
Proof Put $n\equiv\dim{V}=\dim{W}$. Let $e\equiv e_1,\dots,e_n$ be an orthonormal basis of $V$ and let $f\equiv f_1,\dots,f_n$ be orthonormal basis of $W$.
First suppose that $S$ is an isometry. Note that 6.30 gives
$$ Se_k=\sum_{i=1}^n\innprd{Se_k}{f_i}_Wf_i $$
Then
$$\align{ \mtrxof{S,e,f} &= \pmtrx{\mtrxof{S,e,f}_{:,1}&\dotsb&\mtrxof{S,e,f}_{:,n}} \\ &= \pmtrx{\mtrxof{Se_1,f}&\dotsb&\mtrxof{Se_n,f}} \\ &= \pmtrx{\innprd{Se_1}{f_1}_W&\dotsb&\innprd{Se_n}{f_1}_W\\\vdots&\ddots&\vdots\\\innprd{Se_1}{f_n}_W&\dotsb&\innprd{Se_n}{f_n}_W} \\ }$$
and
$$\align{ \innprdbg{\mtrxof{S,e,f}_{:,j}}{\mtrxof{S,e,f}_{:,k}}_{\wF^n} &= \sum_{i=1}^n\innprd{Se_j}{f_i}_W\cj{\innprd{Se_k}{f_i}_W}\tag{by W.6.G.9} \\ &= \sum_{i=1}^n\innprdbg{Se_j}{\innprd{Se_k}{f_i}_Wf_i}_W \\ &= \innprdBgg{Se_j}{\sum_{i=1}^n\innprd{Se_k}{f_i}_Wf_i}_W \\ &= \innprd{Se_j}{Se_k}_W\tag{by 6.30} \\ &= \cases{1&j=k\\0&j\neq k} }$$
The last equality holds because $Se_1,\dots,Se_n$ is orthonormal (by W.7.G.43(c)). Hence the columns of $\mtrxof{S,e,f}$ are orthonormal with respect to $\innprd{\cdot}{\cdot}_{\wF^n}$. Also note that $Se_1,\dots,Se_n$ is an orthonormal basis of $W$ since $n=\dim{W}$ (by 6.28). Hence
$$\align{ \innprdbg{\mtrxof{S,e,f}_{j,:}}{\mtrxof{S,e,f}_{k,:}}_{\wF^n} &= \sum_{i=1}^n\innprd{Se_i}{f_j}_W\cj{\innprd{Se_i}{f_k}_W}\tag{by W.6.G.9} \\ &= \sum_{i=1}^n\cj{\innprd{f_j}{Se_i}_W}\innprd{f_k}{Se_i}_W \\ &= \sum_{i=1}^n\innprdbg{f_k}{\innprd{f_j}{Se_i}_WSe_i}_W \\ &= \innprdBgg{f_k}{\sum_{i=1}^n\innprd{f_j}{Se_i}_WSe_i}_W \\ &= \innprd{f_k}{f_j}_W\tag{by 6.30} \\ &= \cases{1&j=k\\0&j\neq k} }$$
Hence the rows of $\mtrxof{S,e,f}$ are orthonormal with respect to $\innprd{\cdot}{\cdot}_{\wF^n}$.
Conversely, suppose that, for any orthonormal bases $\epsilon$ of $V$ and $\varphi$ of $W$, the columns and rows of $\mtrxof{S,\epsilon,\varphi}$ are orthonormal with respect $\innprddt_{\wF^n}$. Then
$$\align{ \mtrxof{\adjt{S}S,e,e} &= \mtrxof{\adjt{S},f,e}\mtrxof{S,e,f}\tag{by W.3.27} \\ &= \adjt{\mtrxof{S,e,f}}\mtrxof{S,e,f}\tag{by W.7.G.35} \\ &= I_n\tag{by W.7.G.28} \\ &= \mtrxof{I_V,e,e} }$$
Since $\mtrxof{\cdot,e,e}$ is an isomorphism (by 3.60), then it’s injective (by 3.56). Hence $\adjt{S}S=I_V$ and $S$ is an isometry (by W.7.G.43(e)). $\wes$
Change of Basis
Example W.7.CoB.0 Let $u_1,u_2$ and $v_1,v_2$ be bases of $V$ and let $w\in V$. Suppose we’re given the scalars $w_1,w_2$ and $u_{1,1},u_{2,1}$ and $u_{1,2},u_{2,2}$ such that
$$ w=w_1u_1+w_2u_2 \dq\dq u_1=u_{1,1}v_1+u_{2,1}v_2 \dq\dq u_2=u_{1,2}v_1+u_{2,2}v_2 $$
Hence
$$ \mtrxofb{w,(u_1,u_2)}=\pmtrx{w_1\\w_2} \dq\dq \mtrxofb{u_1,(v_1,v_2)}=\pmtrx{u_{1,1}\\u_{2,1}} \dq\dq \mtrxofb{u_2,(v_1,v_2)}=\pmtrx{u_{1,2}\\u_{2,2}} $$
Then how can we compute $\mtrxofb{w,(v_1,v_2)}$? Note that
$$\align{ w &= w_1u_1+w_2u_2 \\ &= w_1(u_{1,1}v_1+u_{2,1}v_2)+w_2(u_{1,2}v_1+u_{2,2}v_2) \\ &= u_{1,1}w_1v_1+u_{2,1}w_1v_2+u_{1,2}w_2v_1+u_{2,2}w_2v_2\tag{W.7.CoB.0.2} \\ &= u_{1,1}w_1v_1+u_{1,2}w_2v_1+u_{2,1}w_1v_2+u_{2,2}w_2v_2 \\ &= (u_{1,1}w_1+u_{1,2}w_2)v_1+(u_{2,1}w_1+u_{2,2}w_2)v_2\tag{W.7.CoB.0.3} }$$
and
$$\align{ \mtrxofb{w,(v_1,v_2)} &= \pmtrx{u_{1,1}w_1+u_{1,2}w_2\\u_{2,1}w_1+u_{2,2}w_2} \\ &= \pmtrx{u_{1,1}&u_{1,2}\\u_{2,1}&u_{2,2}}\pmtrx{w_1\\w_2} \\ &= \pmtrx{\mtrxofb{u_1,(v_1,v_2)} & \mtrxofb{u_2,(v_1,v_2)}}\mtrxofb{w,(u_1,u_2)} \\ &= \pmtrx{\mtrxofb{Iu_1,(v_1,v_2)}&\mtrxofb{Iu_2,(v_1,v_2)}}\mtrxofb{w,(u_1,u_2)} \\ &= \pmtrx{\mtrxofb{I,(u_1,u_2),(v_1,v_2)}_{:,1}&\mtrxofb{I,(u_1,u_2),(v_1,v_2)}_{:,2}}\mtrxofb{w,(u_1,u_2)} \\ &= \mtrxofb{I,(u_1,u_2),(v_1,v_2)}\mtrxofb{w,(u_1,u_2)} }$$
Another way to see this:
$$\align{ \nbmtrx{ Iu_1=u_1=u_{1,1}v_1+u_{2,1}v_2 \\ Iu_2=u_2=u_{1,2}v_1+u_{2,2}v_2 } \dq\iff\dq \pmtrx{u_{1,1}&u_{1,2}\\u_{2,1}&u_{2,2}}=\mtrxofb{I,(u_1,u_2),(v_1,v_2)} }$$
Alternatively, define $C\wiov$ by $Cv_j=u_j$ for $j=1,2$. Then
$$ Cv_1=u_1=u_{1,1}v_1+u_{2,1}v_2 \dq\dq Cv_2=u_2=u_{1,2}v_1+u_{2,2}v_2 $$
and
$$\align{ \mtrxofb{C,(v_1,v_2)} &= \mtrxofb{C,(v_1,v_2),(v_1,v_2)} \\ &= \pmtrx{\mtrxofb{C,(v_1,v_2),(v_1,v_2)}_{:,1}&\mtrxofb{C,(v_1,v_2),(v_1,v_2)}_{:,2}} \\ &= \pmtrx{\mtrxofb{Cv_1,(v_1,v_2)}&\mtrxofb{Cv_2,(v_1,v_2)}} \\ &= \pmtrx{u_{1,1}&u_{1,2}\\u_{2,1}&u_{2,2}} \\ &= \mtrxofb{I,(u_1,u_2),(v_1,v_2)} }$$
Now suppose that $u_1,u_2$ and $v_1,v_2$ are orthonormal bases of $V$. Then
$$\align{ \dnorm{Cw}^2 &= \dnorm{C\prn{\innprd{w}{v_1}v_1+\innprd{w}{v_2}v_2}}^2 \\ &= \dnormb{\innprd{w}{v_1}Cv_1+\innprd{w}{v_2}Cv_2}^2 \\ &= \dnormb{\innprd{w}{v_1}u_1+\innprd{w}{v_2}u_2}^2 \\ &= \dnormb{\innprd{w}{v_1}u_1}^2+\dnormb{\innprd{w}{v_2}u_2}^2 \\ &= \normb{\innprd{w}{v_1}}^2\dnorm{u_1}^2+\normb{\innprd{w}{v_2}}^2\dnorm{u_2}^2 \\ &= \normb{\innprd{w}{v_1}}^2+\normb{\innprd{w}{v_2}}^2 \\ &= \dnorm{w}^2 }$$
Hence $C$ is an isometry. In particular, the matrix $\mtrxofb{C,(v_1,v_2)}=\mtrxofb{I,(u_1,u_2),(v_1,v_2)}$ is orthogonal (if $\wF=\wR$) or unitary (if $\wF=\wC$) (by W.7.G.37).
Proposition W.7.CoB.1 Let $u_1,\dots,u_n$ and $v_1,\dots,v_n$ be bases of $V$. Then, for any $w\in V$, we have
$$ \mtrxofb{w,(v_1,\dots,v_n)}=\mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)}\mtrxofb{w,(u_1,\dots,u_n)} \tag{W.7.CoB.1.1} $$
Also, define $C\wiov$ by
$$ Cv_j=u_j\qd\text{for }j=1,\dots,n \tag{W.7.CoB.1.2} $$
Then
$$ \mtrxofb{C,(v_1,\dots,v_n)}=\mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)} \tag{W.7.CoB.1.3} $$
Proof Since $v_1,\dots,v_n$ is a basis of $V$, then there exist scalars $u_{k,j}$ such that $u_j=\sum_{k=1}^nu_{k,j}v_k$. Since $u_1,\dots,u_n$ is a basis of $V$, then there exist scalars $w_j$ such that $w = \sum_{j=1}^nw_ju_j$. Hence
$$ w = \sum_{j=1}^nw_ju_j = \sum_{j=1}^nw_j\sum_{k=1}^nu_{k,j}v_k = \sum_{j=1}^n\sum_{k=1}^nu_{k,j}w_jv_k = \sum_{k=1}^n\sum_{j=1}^nu_{k,j}w_jv_k = \sum_{k=1}^n\Prn{\sum_{j=1}^nu_{k,j}w_j}v_k $$
Hence
$$\align{ \mtrxofb{w,(v_1,\dots,v_n)} &= \pmtrx{\sum_{j=1}^nu_{1,j}w_j\\\vdots\\\sum_{j=1}^nu_{n,j}w_j} \\ &= \pmtrx{u_{1,1}&\dotsb&u_{1,n}\\\vdots&\ddots&\vdots\\u_{n,1}&\dotsb&u_{n,n}}\pmtrx{w_1\\\vdots\\w_n} \\ &= \pmtrx{\mtrxofb{u_1,(v_1,\dots,v_n)}&\dotsb & \mtrxofb{u_n,(v_1,\dots,v_n)}}\mtrxofb{w,(u_1,\dots,u_n)} \\ &= \pmtrx{\mtrxofb{Iu_1,(v_1,\dots,v_n)}&\dotsb&\mtrxofb{Iu_n,(v_1,\dots,v_n)}}\mtrxofb{w,(u_1,\dots,u_n)} \\ &= \pmtrx{\mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)}_{:,1}&\dotsb&\mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)}_{:,n}}\mtrxofb{w,(u_1,\dots,u_n)} \\ &= \mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)}\mtrxofb{w,(u_1,\dots,u_n)} }$$
Next, note that
$$ Cv_j=u_j=\sum_{k=1}^nu_{k,j}v_k $$
Hence
$$\align{ \mtrxofb{C,(v_1,\dots,v_n)} &= \mtrxofb{C,(v_1,\dots,v_n),(v_1,\dots,v_n)} \\ &= \pmtrx{\mtrxofb{C,(v_1,\dots,v_n),(v_1,\dots,v_n)}_{:,1}&\dotsb&\mtrxofb{C,(v_1,\dots,v_n),(v_1,\dots,v_n)}_{:,n}} \\ &= \pmtrx{\mtrxofb{Cv_1,(v_1,\dots,v_n)}&\dotsb&\mtrxofb{Cv_n,(v_1,\dots,v_n)}} \\ &= \pmtrx{u_{1,1}&\dotsb&u_{1,n}\\\vdots&\ddots&\vdots\\u_{n,1}&\dotsb&u_{n,n}} \\ &= \mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)} }$$
$\wes$
Proposition W.7.CoB.2 Let $u_1,\dots,u_n$ and $v_1,\dots,v_n$ be orthonormal bases of $V$. Define $C\wiov$ as in W.7.CoB.1.2. Then $C$ is an isometry and
$$ Cv_j=\sum_{k=1}^n\innprd{u_j}{v_k}v_k \tag{W.7.CoB.2.1} $$
In particular, the matrix $\mtrxofb{C,(v_1,\dots,v_n)}=\mtrxofb{I,(u_1,\dots,u_n),(v_1,\dots,v_n)}$ is orthogonal (if $\wF=\wR$) or unitary (if $\wF=\wC$) (by W.7.G.37).
Proof W.7.CoB.2.1 follows from 6.30 and $C$ is an isometry because it maps an orthonormal basis $v_j$ to an orthonormal basis $Cv_j=u_j$ (by 7.42(c)). Alternatively, let $w\in V$. Then
$$\align{ \dnorm{Cw}^2 &= \dnorm{C\Prngg{\sum_{j=1}^n\innprd{w}{v_j}v_j}}^2\tag{by 6.30} \\ &= \dnorm{\sum_{j=1}^n\innprd{w}{v_j}Cv_j}^2 \\ &= \dnorm{\sum_{j=1}^n\innprd{w}{v_j}u_j}^2 \\ &= \sum_{j=1}^n\normb{\innprd{w}{v_j}}^2\tag{by 6.25} \\ &= \dnorm{w}^2\tag{by 6.30} }$$
Hence $C$ is an isometry. For yet another proof, we can show that $C$ is an isometry by 7.42(e):
$$\align{ \mtrxofb{\adjt{C}C,(v_1,\dots,v_n)} &= \mtrxofb{\adjt{C},(v_1,\dots,v_n)}\mtrxofb{C,(v_1,\dots,v_n)} \\\\ &= \adjt{\mtrxofb{C,(v_1,\dots,v_n)}}\mtrxofb{C,(v_1,\dots,v_n)}\tag{by 7.10} \\\\ &= \pmtrx{\cj{\innprd{u_1}{v_1}}&\dotsb&\cj{\innprd{u_1}{v_n}}\\\vdots&\ddots&\vdots\\\cj{\innprd{u_n}{v_1}}&\dotsb&\cj{\innprd{u_n}{v_n}}} \pmtrx{\innprd{u_1}{v_1}&\dotsb&\innprd{u_n}{v_1}\\\vdots&\ddots&\vdots\\\innprd{u_1}{v_n}&\dotsb&\innprd{u_n}{v_n}} \\\\ &= \pmtrx{\sum_{j=1}^n\cj{\innprd{u_1}{v_j}}\innprd{u_1}{v_j}&\dotsb&\sum_{j=1}^n\cj{\innprd{u_1}{v_j}}\innprd{u_n}{v_j} \\ \vdots&\ddots&\vdots\\ \sum_{j=1}^n\cj{\innprd{u_n}{v_j}}\innprd{u_1}{v_j}&\dotsb&\sum_{j=1}^n\cj{\innprd{u_n}{v_j}}\innprd{u_n}{v_j}} \\\\ &= \pmtrx{\sum_{j=1}^n\normb{\innprd{u_1}{v_j}}^2&\dotsb&\sum_{j=1}^n\innprdbg{u_n}{\innprd{u_1}{v_j}v_j} \\ \vdots&\ddots&\vdots\\ \sum_{j=1}^n\innprdbg{u_1}{\innprd{u_n}{v_j}v_j}&\dotsb&\sum_{j=1}^n\normb{\innprd{u_n}{v_j}}^2} \\\\ &= \pmtrx{\dnorm{u_1}^2&\dotsb&\innprdBg{u_n}{\sum_{j=1}^n\innprd{u_1}{v_j}v_j} \\ \vdots&\ddots&\vdots\\ \innprdBg{u_1}{\sum_{j=1}^n\innprd{u_n}{v_j}v_j}&\dotsb&\dnorm{u_n}^2}\tag{by 6.30} \\\\ &= \pmtrx{1&\dotsb&\innprd{u_n}{u_1}\\\vdots&\ddots&\vdots\\\innprd{u_1}{u_n}&\dotsb&1}\tag{by 6.30} \\\\ &= \pmtrx{1&\dotsb&0\\\vdots&\ddots&\vdots\\0&\dotsb&1} \\\\ &= \mtrxofb{I,(v_1,\dots,v_n)} }$$
Since $\mtrxofsb\in\linmapb{\oper{V}}{\wF^{n,n}}$ is an isomorphism (by 3.60), then it’s also injective (by 3.69). Hence $\adjt{C}C=I$. Hence $C$ is an isometry (by 7.42(e)). $\wes$
Example W.7.CoB.3 Define
$$ u_1\equiv(1,0,0) \dq u_2\equiv(0,1,0) \dq u_3\equiv(0,0,1) $$
$$ v_1\equiv(\cos{\theta},0,-\sin{\theta}) \dq v_2\equiv(0,1,0) \dq v_3\equiv(\sin{\theta},0,\cos{\theta}) $$
Then $u_1,u_2,u_3$ and $v_1,v_2,v_3$ are orthonormal bases of $\wR^3$ under the dot product. Let $(x,y,z)\in\wR^3$ so that
$$\align{ \mtrxofb{(x,y,z),(u_1,u_2,u_3)} &= \pmtrx{\innprdbg{(x,y,z)}{u_1}\\\innprdbg{(x,y,z)}{u_2}\\\innprdbg{(x,y,z)}{u_3}}=\pmtrx{x\\y\\z} \\\\ \mtrxofb{(x,y,z),(v_1,v_2,v_3)} &= \pmtrx{\innprdbg{(x,y,z)}{v_1}\\\innprdbg{(x,y,z)}{v_2}\\\innprdbg{(x,y,z)}{v_3}}=\pmtrx{x\cos{\theta}-z\sin{\theta}\\y\\x\sin{\theta}+z\cos{\theta}} \tag{W.7.CoB.3.1} }$$
To convert between the two bases, we use the change of basis formula from W.7.CoB.2.1:
$$\align{ Cv_1 &= \sum_{k=1}^3\innprd{u_1}{v_k}v_k \\ &= \innprdbg{(1,0,0)}{(\cos{\theta},0,-\sin{\theta})}v_1+\innprdbg{(1,0,0)}{(0,1,0)}v_2+\innprdbg{(1,0,0)}{(\sin{\theta},0,\cos{\theta})}v_3 \\ &= (\cos{\theta})v_1+0v_2+(\sin{\theta})v_3 \\\\ Cv_2 &= \sum_{k=1}^3\innprd{u_2}{v_k}v_k \\ &= \innprdbg{(0,1,0)}{(\cos{\theta},0,-\sin{\theta})}v_1+\innprdbg{(0,1,0)}{(0,1,0)}v_2+\innprdbg{(0,1,0)}{(\sin{\theta},0,\cos{\theta})}v_3 \\ &= 0v_1+1v_2+0v_3 \\\\ Cv_3 &= \sum_{k=1}^3\innprd{u_3}{v_k}v_k \\ &= \innprdbg{(0,0,1)}{(\cos{\theta},0,-\sin{\theta})}v_1+\innprdbg{(0,0,1)}{(0,1,0)}v_2+\innprdbg{(0,0,1)}{(\sin{\theta},0,\cos{\theta})}v_3 \\ &= (-\sin{\theta})v_1+0v_2+(\cos{\theta})v_3 }$$
Put $v\equiv v_1,v_2,v_3$. Then
$$\align{ \mtrxofb{C,(v_1,v_2,v_3)} &= \mtrxofb{C,(v_1,v_2,v_3),(v_1,v_2,v_3)} \\ &= \mtrxof{C,v,v} \\ &= \pmtrx{\mtrxof{C,v,v}_{:,1}&\mtrxof{C,v,v}_{:,2}&\mtrxof{C,v,v}_{:,3}} \\ &= \pmtrx{\mtrxof{Cv_1,v}&\mtrxof{Cv_2,v}&\mtrxof{Cv_3,v}} \\ &= \pmtrx{\costht&0&-\sintht\\0&1&0\\\sintht&0&\costht} }$$
and
$$\align{ \mtrxofb{(x,y,z),(v_1,v_2,v_3)} &= \mtrxofb{I,(u_1,u_2,u_3),(v_1,v_2,v_3)}\mtrxofb{(x,y,z),(u_1,u_2,u_3)}\tag{by W.7.CoB.1.1} \\ &= \mtrxofb{C,(v_1,v_2,v_3)}\mtrxofb{(x,y,z),(u_1,u_2,u_3)}\tag{by W.7.CoB.1.3} \\ &= \pmtrx{\costht&0&-\sintht\\0&1&0\\\sintht&0&\costht}\pmtrx{x\\y\\z} \\ &= \pmtrx{x\cos{\theta}-z\sin{\theta}\\y\\x\sin{\theta}+z\cos{\theta}} }$$
This agrees with W.7.CoB.3.1. $\wes$
Proposition W.7.CoB.5 <–> 10.7 Suppose $T\wiov$. Let $u\equiv u_1,\dots,u_n$ and $v\equiv v_1,\dots,v_n$ be bases of $V$. Then
$$\align{ \mtrxof{T,u} &= \mtrxof{I,v,u}\mtrxof{T,v}\mtrxof{I,u,v} \\ &= \inv{\mtrxof{I,u,v}}\mtrxof{T,v}\mtrxof{I,u,v} }$$
Proof
$$\align{ \inv{\mtrxof{I,u,v}}\mtrxof{T,v}\mtrxof{I,u,v} &= \mtrxof{I,v,u}\mtrxof{T,v,v}\mtrxof{I,u,v}\tag{by W.3.28} \\ &= \mtrxof{IT,v,u}\mtrxof{I,u,v}\tag{by W.3.27} \\ &= \mtrxof{T,v,u}\mtrxof{I,u,v} \\ &= \mtrxof{TI,u,u}\tag{by W.3.27} \\ &= \mtrxof{T,u} }$$
$\wes$
Polar Decomposition
Proposition W.7.PLRDC.1 Let $T\in\linmap{V}{W}$. Then $\dnorm{Tv}_W=\dnorm{\sqrttat v}_V$ for all $v\in V$.
Proof For any $v\in V$, we have
$$\align{ \dnorm{Tv}_W^2 &= \innprd{Tv}{Tv}_W \\ &= \innprd{\tat v}{v}_V \\ &= \innprdbg{\sqrttat\sqrttat v}{v}_V \\ &= \innprdbg{\sqrttat v}{\adjt{\prn{\sqrttat}} v}_V \\ &= \innprdbg{\sqrttat v}{\sqrttat v}_V \\ &= \dnorm{\sqrttat v}_V^2 }$$
The next-to-last equality holds because $\sqrttat$ is positive and hence self-adjoint. $\wes$
Proposition W.7.PLRDC.2 Let $T\in\linmap{V}{W}$. Then
$$ \nullsp{T}=\nullspb{\sqrttat}=\rangspb{\sqrttat}^\perp $$
Proof Let $u\in\nullsp{T}$. Then 6.10(a) gives the first equality and W.7.PLRDC.1 gives the second equality:
$$ 0=\dnorm{Tu}_W=\dnorm{\sqrttat u}_V $$
Hence, by 6.10(a), $\sqrttat u=0$. Hence $u\in\nullspb{\sqrttat}$ and $\nullsp{T}\subset\nullspb{\sqrttat}$.
Conversely, let $u\in\nullspb{\sqrttat}$. Then
$$ 0=\dnorm{\sqrttat u}_V=\dnorm{Tu}_W $$
Hence $Tu=0$ and $u\in\nullsp{T}$ and $\nullspb{\sqrttat}\subset\nullsp{T}$. Hence $\nullspb{\sqrttat}=\nullsp{T}$. Also note that
$$ \nullspb{\sqrttat}=\nullspb{\adjt{\prn{\sqrttat}}}=\rangspb{\sqrttat}^\perp $$
The first equality holds because $\sqrttat$ is positive and hence self-adjoint. The second equality follows from 7.7(a). $\wes$
Proposition W.7.PLRDC.3 Let $T\in\linmap{V}{W}$ and let $f_1,\dots,f_n$ denote an orthonormal basis of $V$ consisting of eigenvectors of $\tat$ corresponding to eigenvalues $\lambda_1,\dots,\lambda_n$ (existence given by the Spectral Theorem). W.7.G.21 and 7.35(b) imply that all of the $\lambda$’s are nonnegative. Without loss of generality, assume that $\lambda_1,\dots,\lambda_r$ are positive and that $\lambda_{r+1},\dots,\lambda_n$ are zero (re-index if necessary). Then
(A) $f_1,\dots,f_r$ is an orthonormal basis of $\rangspb{\sqrttat}$
(B) $f_{r+1},\dots,f_n$ is an orthonormal basis of $\rangspb{\sqrttat}^\perp=\nullspb{\sqrttat}=\nullsp{T}$
(C) $\rangsp{T}=\span{Tf_1,\dots,Tf_r}$
(D) $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r$ is an orthonormal basis of $\rangsp{T}$
(E) $\dim{\rangspb{\sqrttat}}=\dim{\rangsp{T}}=r$
(F) If $\dim{W}\geq\dim{V}$, then $\dim{\rangsp{T}^\perp}\geq\dim{V}-r$
(G) $\adjtT h=0$ for all $h\in\rangsp{T}^\perp$
Proof of (A) & (B) The previous proposition gives $\nullspb{\sqrttat}=\rangspb{\sqrttat}^\perp=\nullsp{T}$. So to prove (B), it suffices to show that $f_{r+1},\dots,f_n$ is an orthonormal basis of $\nullspb{\sqrttat}$.
Note that
$$ \tat f_j\equiv\lambda_j f_j\qd\text{for }j=1,\dots,n $$
W.7.G.21 gives
$$ \sqrttat f_j\equiv\sqrt{\lambda_j}f_j\qd\text{for }j=1,\dots,n $$
In particular, for $j=r+1,\dots,n$, we have $\sqrttat f_j=0\cdot f_j=0$. Hence
$$ f_{r+1},\dots,f_n\in\nullspb{\sqrttat} $$
Also note that exercise 5.C.7 gives
$$ \dim{\nullspb{\sqrttat}}=\dim{\eignspb{0}{\sqrttat}}=\text{number of }\lambda_j\text{'s equal to }0=n-r $$
Hence $f_{r+1},\dots,f_n$ is an orthonormal basis of $\nullspb{\sqrttat}$ (by 6.28). Hence
$$ \nullspb{\sqrttat}=\span{f_{r+1},\dots,f_n} $$
Also, since $f_1,\dots,f_n$ is orthonormal, then $f_j\in\setb{f_{r+1},\dots,f_n}^\perp$ for $j=1,\dots,r$. Then exercise 6.C.1 gives the first equality:
$$ f_j\in\setb{f_{r+1},\dots,f_n}^\perp=\prn{\span{f_{r+1},\dots,f_n}}^\perp=\nullspb{\sqrttat}^\perp\qd\text{for }j=1,\dots,r $$
Also note that W.7.G.15 implies that $\sqrttat$ is positive. Hence $\sqrttat$ is self-adjoint. This gives the second equality and the first equality follows from 7.7(b):
$$ f_1,\dots,f_r\in\nullspb{\sqrttat}^\perp = \rangspb{\adjt{\prn{\sqrttat}}} = \rangspb{\sqrttat} $$
Lastly, the Fundamental Theorem of Linear Maps (3.22) gives
$$ \dim{\rangspb{\sqrttat}}=\dim{V}-\dim{\nullspb{\sqrttat}}=n-(n-r)=r $$
Hence $f_1,\dots,f_r$ is an orthonormal basis of $\rangspb{\sqrttat}$ (by 6.28). $\wes$
Proof of (C) & (D) & (E) Part (B) gives $Tf_j=0$ for $j=r+1,\dots,n$.
Let $w\in\rangsp{T}$. Then $w=Tv$ for some $v\in V$ and $v=\sum_{j=1}^n\innprd{v}{f_j}f_j$ (by 6.30). Hence
$$ w=Tv=T\Prngg{\sum_{j=1}^n\innprd{v}{f_j}f_j}=\sum_{j=1}^n\innprd{v}{f_j}Tf_j=\sum_{j=1}^r\innprd{v}{f_j}Tf_j\in\span{Tf_1,\dots,Tf_r} $$
Hence $\rangsp{T}\subset\span{Tf_1,\dots,Tf_r}$. Conversely, we have $Tf_1,\dots,Tf_r\in\rangsp{T}$. Since $\rangsp{T}$ is closed under addition and scalar multiplication, then $\rangsp{T}$ contains any linear combination of $Tf_1,\dots,Tf_r$. Hence $\span{Tf_1,\dots,Tf_r}\subset\rangsp{T}$.
Also:
$$\align{ \innprdBgg{\frac1{\sqrt{\lambda_j}}Tf_j}{\frac1{\sqrt{\lambda_k}}Tf_k}_W &= \frac1{\sqrt{\lambda_j}}\frac1{\sqrt{\lambda_k}}\innprdbg{Tf_j}{Tf_k}_W \\ &= \frac1{\sqrt{\lambda_j}}\frac1{\sqrt{\lambda_k}}\innprdbg{f_j}{\tat f_k}_V \\ &= \frac1{\sqrt{\lambda_j}}\frac1{\sqrt{\lambda_k}}\innprdbg{f_j}{\lambda_kf_k}_V \\ &= \frac1{\sqrt{\lambda_j}}\frac{\lambda_k}{\sqrt{\lambda_k}}\innprd{f_j}{f_k}_V \\ &= \frac{\sqrt{\lambda_k}}{\sqrt{\lambda_j}}\innprd{f_j}{f_k}_V \\ &= \cases{1&j=k\\0&j\neq k} }$$
Hence $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r$ is an orthonormal list in $\rangsp{T}$. By 6.26, $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r$ is linearly independent. By part (C), $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r$ spans $\rangsp{T}$. Hence $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r$ is an orthonormal basis of $\rangsp{T}$.
Hence $\dim{\rangspb{\sqrttat}}=\dim{\rangsp{T}}=r$. $\wes$
Proof of (F) & (G) 6.50 gives
$$ \dim{\rangsp{T}^\perp}=\dim{W}-\dim{\rangsp{T}}\geq \dim{V}-r $$
and W.7.CoA.1 gives
$$ \adjtT h=\sum_{j=1}^n\innprd{h}{Tf_j}f_j=0 $$
since $h\in\rangsp{T}^\perp$ and $Tf_j\in\rangsp{T}$ for $j=1,\dots,n$. $\wes$
Example W.7.PLRDC.3.a Define $T\in\linmap{\wR}{\wR^2}$ by
$$ Te_1\equiv g_1+g_2 \dq\text{or}\dq \mtrxof{T,e,g}\equiv\pmtrx{1\\1} $$
where $e\equiv e_1=(1)$ is the standard basis of $\wR$ and $g\equiv g_1,g_2$ is the standard basis of $\wR^2$. Then
$$ Tx=T(xe_1)=xTe_1=x(g_1+g_2)=xg_1+xg_2=(x,x) \dq\text{or}\dq \mtrxof{T,e,g}\pmtrx{x}=\pmtrx{1\\1}\pmtrx{x}=\pmtrx{x\\x} $$
Hence $\rangsp{T}=\setb{(x,x)\in\wR^2:x\in\wR}$. Hence $\rangsp{T}^\perp=\setb{(x,-x)\in\wR^2:x\in\wR}$. Also
$$ \innprdbg{(x)}{\adjtT(y_1,y_2)}_{\wR} = \innprdbg{T(x)}{(y_1,y_2)}_{\wR^2} = \innprdbg{(x,x)}{(y_1,y_2)}_{\wR^2} = xy_1+xy_2 = \innprdbg{(x)}{(y_1+y_2)}_{\wR} $$
Hence $\adjtT(y_1,y_2)=y_1+y_2$. Hence $\tat x=\adjtT(x,x)=2x$. Or
$$ \mtrxof{\tat,e,e}=\mtrxof{\adjtT,g,e}\mtrxof{T,e,g}=\adjt{\mtrxof{T,e,g}}\mtrxof{T,e,g}=\pmtrx{1&1}\pmtrx{1\\1}=\pmtrx{2} $$
So $e_1$ is an orthonormal basis of $\wR$ consisting of eigenvectors of $\tat$ corresponding to eigenvalue $\lambda_1=2$. And
$$ \frac1{\sqrt2}Te_1=\frac1{\sqrt2}(g_1+g_2)=\frac1{\sqrt2}(1,1)=\Prn{\frac1{\sqrt2},\frac1{\sqrt2}} $$
is an orthonormal basis of $\rangsp{T}$. Note that $\dim{\rangsp{T}}=1$ and $\dim{\rangsp{T}}^\perp=1\geq 0=\dim{\wR}-\dim{\rangsp{T}}$.
Proposition W.7.PLRDC.4 Let $T\in\linmap{V}{W}$ and let $f_1,\dots,f_n$ denote an orthonormal basis of $V$ consisting of eigenvectors of $\tat$ corresponding to eigenvalues $\lambda_1,\dots,\lambda_n$ (existence given by the Spectral Theorem). And let $f_1,\dots,f_r$ denote the sublist that is an orthonormal basis of $\rangspb{\sqrttat}$ (existence given by W.7.PLRDC.3).
Define $S_1\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ by
$$ S_1f_j\equiv\frac1{\sqrt{\lambda_j}}Tf_j\qd\text{for }j=1,\dots,r $$
And define $S_1’\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ by
$$ S_1'\prn{\sqrttat v}\equiv Tv\qd\text{for all }v\in V $$
Then $S_1=S_1’$ and $\dnorm{S_1\alpha}_W=\dnorm{\alpha}_V$ for all $\alpha\in\rangspb{\sqrttat}$.
Proof W.7.PLRDC.3 shows that $\sqrt{\lambda_j}$, the eigenvalue of $\sqrttat$ corresponding to $f_j$, is positive for $j=1,\dots,r$. Since $f_1,\dots,f_r$ is a basis of $\rangspb{\sqrttat}$ (by W.7.PLRDC.3.A), then $S_1$ is a well-defined linear map (by 3.5).
We must show that $S_1’$ is a well-defined function. Suppose $v_1,v_2\in V$ are such that $\sqrttat v_1=\sqrttat v_2$. If it’s the case $Tv_1\neq Tv_2$, then $S_1’$ is not well-defined. But we have
$$ \dnorm{Tv_1-Tv_2}_W=\dnorm{T(v_1-v_2)}_W=\dnorm{\sqrttat(v_1-v_2)}_V=\dnorm{\sqrttat v_1-\sqrttat v_2}_V=0 $$
where the second equality follows from W.7.PLRDC.1. Hence, by 6.10(a), $Tv_1=Tv_2$.
$S_1’$ is also linear: Let $u_1,u_2\in\rangspb{\sqrttat}$. Then there exist $v_1,v_2\in V$ such that $u_1=\sqrttat v_1$ and $u_2=\sqrttat v_2$. Also let $\lambda\in\wF$. Then
$$\align{ S_1'(\lambda u_1+u_2) &= S_1'\prn{\lambda\sqrttat v_1+\sqrttat v_2} \\ &= S_1'\prn{\sqrttat(\lambda v_1)+\sqrttat v_2} \\ &= S_1'\prn{\sqrttat(\lambda v_1+v_2)} \\ &= T(\lambda v_1+v_2) \\ &= T(\lambda v_1)+Tv_2 \\ &= \lambda Tv_1+Tv_2 \\ &= \lambda S_1'\prn{\sqrttat v_1}+S_1'\prn{\sqrttat v_2} \\ &= \lambda S_1'u_1+S_1'u_2 }$$
Hence, for $j=1,\dots,r$, we have
$$ S_1'f_j = \frac{\sqrt{\lambda_j}}{\sqrt{\lambda_j}}S_1'f_j = \frac1{\sqrt{\lambda_j}}S_1'\prn{\sqrt{\lambda_j}f_j} = \frac1{\sqrt{\lambda_j}}S_1'\prn{\sqrttat f_j} = \frac1{\sqrt{\lambda_j}}Tf_j = S_1f_j $$
Since $f_1,\dots,f_r$ is a basis of $\rangspb{\sqrttat}$, then $S_1=S_1’$ (by 3.5).
Lastly, let $\alpha\in\rangspb{\sqrttat}$. Then there exists $v\in V$ such that $\alpha=\sqrttat v$. Hence
$$ \dnorm{S_1\alpha}_W=\dnorm{S_1'\alpha}_W=\dnorm{S_1'\prn{\sqrttat v}}_W=\dnorm{Tv}_W=\dnorm{\sqrttat v}_V=\dnorm{\alpha}_V $$
where the next-to-last equality follows from W.7.PLRDC.1. $\wes$
Proposition W.7.PLRDC.5 Let $T\in\linmap{V}{W}$.
If $\dim{W}\geq\dim{V}$, then there exist a subspace $U$ of $W$ with $\dim{U}=\dim{V}$, an isometry $S\in\linmap{V}{U}$, and an orthonormal basis $f\equiv f_1,\dots,f_n$ of $V$ consisting of eigenvectors of $\tat$ such that
$$ Tv=S\sqrttat v\qd\text{for all }v\in V \tag{W.7.PLRDC.5.1} $$
and
$$ Sf_j=\cases{\frac1{\sqrt{\lambda_j}}Tf_j&\text{for }j=1,\dots,\dim{\rangsp{T}}\\h_j&\text{for }j=\dim{\rangsp{T}}+1,\dots,n} \tag{W.7.PLRDC.5.2} $$
where $h_{\dim{\rangsp{T}}+1},\dots,h_n$ is any orthonormal list in $\rangsp{T}^\perp$ of length $n-\dim{\rangsp{T}}$.
If $\dim{W}\leq\dim{V}$, then there exist a subspace $U$ of $V$ with $\dim{U}=\dim{W}$ and an isometry $S\in\linmap{U}{W}$ such that
$$ Tv=\sqrttta Sv\qd\text{for all }v\in V $$
Proof First suppose that $\dim{W}\geq\dim{V}$.
Let $f_1,\dots,f_n$ denote an orthonormal basis of $V$ consisting of eigenvectors of $\tat$ corresponding to eigenvalues $\lambda_1,\dots,\lambda_n$ (existence given by the Spectral Theorem). And let $f_1,\dots,f_r$ denote the sublist that is an orthonormal basis of $\rangspb{\sqrttat}$ (existence given by W.7.PLRDC.3 - re-index if necessary) so that $\lambda_1,\dots,\lambda_r$ are positive. By W.7.PLRDC.3(e), we have $r=\dim{\rangsp{T}}$. Define $S_1\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ by
$$ S_1f_j\equiv\frac1{\sqrt{\lambda_j}}Tf_j\qd\text{for }j=1,\dots,r $$
By W.7.PLRDC.3(B), $f_{r+1},\dots,f_n$ is an orthonormal basis of $\rangspb{\sqrttat}^\perp$. Since $\dim{W}\geq\dim{V}$, then W.7.PLRDC.3(F) implies that $\dim{\rangsp{T}}^\perp\geq n-r$. Let $h_{r+1},\dots,h_n$ be an orthonormal list in $\rangsp{T}^\perp$. Define $S_2\in\linmapb{\rangspb{\sqrttat}^\perp}{\span{h_{r+1},\dots,h_n}}$ by
$$ S_2f_j\equiv h_j\qd\text{for }j=r+1,\dots,n $$
Note that for any $\beta\in\rangspb{\sqrttat}^\perp$, we have $\beta=\sum_{j=r+1}^n\innprd{\beta}{f_j}f_j$ and
$$ \dnorm{S_2\beta}_W^2=\dnorm{S_2\Prngg{\sum_{j=r+1}^n\innprd{\beta}{f_j}f_j}}_W^2 =\dnorm{\sum_{j=r+1}^n\innprd{\beta}{f_j}S_2f_j}_W^2 =\dnorm{\sum_{j=r+1}^n\innprd{\beta}{f_j}h_j}_W^2 =\sum_{j=r+1}^n\normw{\innprd{\beta}{f_j}}^2 =\dnorm{\beta}_V^2 $$
The next-to-last equality follows from 6.25 and the last equality follows from 6.30. Put
$$ U \equiv \spanB{\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r,h_{r+1},\dots,h_n} $$
By PLRDC.1, $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r$ is an orthonormal basis of $\rangsp{T}$. And, by construction, $h_{r+1},\dots,h_n$ is an orthonormal list in $\rangsp{T}^\perp$. Hence $\frac1{\sqrt{\lambda_1}}Tf_1,\dots,\frac1{\sqrt{\lambda_r}}Tf_r,h_{r+1},\dots,h_n$ is orthonormal and hence linearly independent. Hence $\dim{U}=n=\dim{V}$.
By 6.47, $V=\rangspb{\sqrttat}\oplus\rangspb{\sqrttat}^\perp$. Hence any $v\in V$ can be uniquely decomposed as $v=\alpha+\beta$ where $\alpha\in\rangspb{\sqrttat}$ and $\beta\in\rangspb{\sqrttat}^\perp$. Define $S\in\linmap{V}{U}$ by
$$ Sv\equiv S_1\alpha+S_2\beta $$
Then
$$ \dnorm{Sv}_W^2=\dnorm{S_1\alpha+S_2\beta}_W^2=\dnorm{S_1\alpha}_W^2+\dnorm{S_2\beta}_W^2=\dnorm{\alpha}_V^2+\dnorm{\beta}_V^2=\dnorm{\alpha+\beta}_V^2=\dnorm{v}_V^2 $$
The second equality holds because $S_1\alpha\in\rangsp{T}$ and $S_2\beta\in\rangsp{T}^\perp$.
Next, define $S_1’$ as in W.7.PLRDC.4. Then, for any $v\in V$, we have
$$ S\sqrttat v = S\prn{\sqrttat v} = S_1\prn{\sqrttat v} = S_1'\prn{\sqrttat v} = Tv $$
The second equality holds because $\sqrttat v\in\rangspb{\sqrttat}$ and the third equality holds because $S_1=S_1’$ (by W.7.PLRDC.4). Also note that
$$ Sf_j=\cases{S_1f_j+S_2(0)&\text{for }j=1,\dots,r\\S_1(0)+S_2f_j&\text{for }j=r+1,\dots,n}=\cases{\frac1{\sqrt{\lambda_j}}Tf_j&\text{for }j=1,\dots,r\\h_j&\text{for }j=r+1,\dots,n}\in U $$
Now suppose that $\dim{W}\leq\dim{V}$. Since $\adjtT\in\linmap{W}{V}$, then there exist a subspace $U$ of $V$ with $\dim{U}=\dim{W}$ and an isometry $\adjt{S}\in\linmap{W}{U}$ such that
$$ \adjtT w=\adjt{S}\sqrt{\adjt{(\adjtT)}\adjtT}w=\adjt{S}\sqrttta w $$
for all $w\in W$. Hence, for all $v\in V$, we have
$$ Tv=\adjt{(\adjtT)}v=\adjt{\prn{\adjt{S}\sqrttta}}v=\adjt{\prn{\sqrttta}}\adjt{(\adjt{S})}v=\sqrttta Sv $$
By 7.42, $S\in\linmap{U}{W}$ is also an isometry. $\wes$
SVD
Proposition W.7.SVD.1 <–> Real (Complex) Spectral Theorem Let $T\wiov$. Then $T$ is self-adjoint (normal) if and only if $V$ has an orthonormal basis $f$ consisting of eigenvectors of $T$ such that, for any orthonormal basis $e$ of $V$, we have
$$ \mtrxof{T,e}=\mtrxof{I,f,e}\mtrxof{T,f}\adjt{\mtrxof{I,f,e}} $$
where $\mtrxof{T,f}$ is diagonal and $\mtrxof{I,f,e}$ is orthogonal (unitary).
Note If $\mtrxof{T,f}$ is diagonal, then 5.32 implies that the eigenvalues of $T$ are precisely the entries on the diagonal of $\mtrxof{T,f}$. And exercise 5.C.7 implies that each eigenvalue $\lambda$ of $T$ appears on the diagonal precisely $\dim{\eignsp{\lambda}{T}}$ times.
Proof Suppose $T$ is self-adjoint (normal). By 7.24(c) or 7.29(c), $T$ has a diagonal matrix with respect to some orthonormal basis $f$ of $V$. Hence, for any orthonormal basis $e$ of $V$, we have
$$\align{ \mtrxof{T,e} &= \mtrxof{I,f,e}\mtrxof{T,f}\mtrxof{I,e,f}\tag{W.7.CoB.5} \\ &= \mtrxof{I,f,e}\mtrxof{T,f}\mtrxof{\adjt{I},e,f}\tag{by 7.6(d)} \\ &= \mtrxof{I,f,e}\mtrxof{T,f}\adjt{\mtrxof{I,f,e}}\tag{by W.7.G.35} }$$
W.7.G.44 implies that $\mtrxof{I,f,e}$ has orthonormal columns with respect to $\innprddt_{\wF^n}$. Hence, by W.7.G.30, $\mtrxof{I,f,e}$ is orthogonal (unitary).
Conversely, suppose that $V$ has an orthonormal basis $f$ consisting of eigenvectors of $T$ such that, for any orthonormal basis $e$ of $V$, we have
$$ \mtrxof{T,e}=\mtrxof{I,f,e}\mtrxof{T,f}\adjt{\mtrxof{I,f,e}} $$
and $\mtrxof{T,f}$ is diagonal. Hence, by 7.24(c) or 7.29(c), $T$ is self-adjoint (normal).
$\wes$
Proposition W.7.SVD.2 Let $T\wiov$ and let $e\equiv e_1,\dots,e_n$ denote any orthonormal basis of $V$. Then there exists an isometry $S\wiov$ and an orthonormal basis $f$ of $V$ such that
$$ \mtrxof{T,e}=\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} \tag{A} $$
where $\mtrxofb{\sqrttat,f}$ is diagonal:
$$ \mtrxofb{\sqrttat,f}=\pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} $$
Additionally:
(B) $\mtrxof{S,f,e}$ and $\mtrxof{I,f,e}$ have orthonormal columns with respect to $\innprddt_{\wF^n}$ and are orthogonal (unitary).
(C) The eigenvalues of $\sqrttat$ are precisely the diagonal entries of $\mtrxofb{\sqrttat,f}$, with each eigenvalue $\lambda_j$ repeated $\dim{\eignspb{\lambda_j}{\sqrttat}}$ times.
(D) The singular values of $T$ are precisely the diagonal entries of $\mtrxofb{\sqrttat,f}$, with each singular value $\lambda_j$ repeated $\dim{\eignspb{\lambda_j}{\sqrttat}}$ times.
(E) The columns of $\mtrxof{I,f,e}$ are eigenvectors of $\mtrxof{\tat,e}$. That is, $\mtrxof{f_j,e}$ is an eigenvector of $\mtrxof{\tat,e}$ corresponding to eigenvalue $\lambda_j^2$.
(F) The columns of $\mtrxof{S,f,e}$ are eigenvectors of $\mtrxof{\tta,e}$. That is, $\mtrxof{Sf_j,e}$ is an eigenvector of $\mtrxof{\tta,e}$ corresponding to eigenvalue $\lambda_j^2$.
(G) $\mtrxof{T,e}=\sum_{j=1}^n\lambda_j\mtrxof{Sf_j,e}\adjt{\mtrxof{f_j,e}}$
Proof of (A) Note that $\tat$ is a positive operator:
$$ \adjt{(\tat)}=\adjtT\adjt{(\adjtT)}=\tat $$
And for any $v\in V$:
$$ \innprd{\tat v}{v}=\innprd{Tv}{\adjt{(\adjtT)}v}=\innprd{Tv}{Tv}=\dnorm{Tv}^2\geq0 $$
Hence, by 7.36 and 7.44, $\sqrttat$ is the unique positive square root of $\tat$. Since $\sqrttat$ is positive, it’s also self-adjoint. Then W.7.SVD.1 gives an orthonormal basis $f$ of $V$ such that $\mtrxofb{\sqrttat,f}$ is diagonal and
$$ \mtrxofb{\sqrttat,e}=\mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} $$
Hence
$$\align{ \mtrxof{T,e} &= \mtrxofb{S\sqrttat,e}\tag{by Polar Decomposition 7.45} \\ &= \mtrxof{S,e}\mtrxofb{\sqrttat,e}\tag{by W.3.27} \\ &= \mtrxof{I,f,e}\mtrxof{S,f}\mtrxof{I,e,f}\mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\tag{by W.7.CoB.5 and W.7.SVD.1} \\ &= \mtrxof{I,f,e}\mtrxof{S,f,f}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\tag{by W.3.28} \\ &= \mtrxof{IS,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\tag{by W.3.27} \\ &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} }$$
Proof of (B) W.7.G.36 implies that $\mtrxof{S,f,e}$ and $\mtrxof{I,f,e}$ have orthonormal columns with respect to $\innprddt_{\wF^n}$. Hence, by W.7.G.30, $\mtrxof{S,f,e}$ and $\mtrxof{I,f,e}$ are orthogonal (unitary).
Proof of (C) W.7.SVD.1, Note 1 implies that the eigenvalues of $\sqrttat$ are precisely the diagonal entries of $\mtrxofb{\sqrttat,f}$, with each eigenvalue $\lambda$ repeated $\dim{\eignspb{\lambda}{\sqrttat}}$ times.
Proof of (D) W.7.G.22 implies that the singular values of $T$ are precisely the diagonal entries of $\mtrxofb{\sqrttat,f}$, with each singular value $\lambda$ repeated $\dim{\eignspb{\lambda}{\sqrttat}}$ times.
Proof of (E) Note that $f_j$ was chosen to be an eigenvector of $\sqrttat$ corresponding to eigenvalue $\lambda_j$. Hence
$$ \tat f_j = \sqrttat\sqrttat f_j = \sqrttat(\lambda_jf_j) = \lambda_j\sqrttat f_j = \lambda_j^2f_j $$
and
$$ \mtrxof{\tat,e}\mtrxof{f_j,e}=\mtrxof{\tat f_j,e}=\mtrxof{\lambda_j^2f_j,e}=\lambda_j^2\mtrxof{f_j,e} $$
Alternatively, note that
$$ \mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}=\pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} $$
And let $\epsilon_1,\dots,\epsilon_n$ denote the standard basis in $\wR^n$. Then
$$\align{ \mtrxof{\tat,e}\mtrxof{f_j,e} &= \mtrxof{\adjtT,e}\mtrxof{T,e}\mtrxof{f_j,e} \\\\ &= \adjt{\mtrxof{T,e}}\mtrxof{T,e}\mtrxof{f_j,e} \\\\ &= \adjt{\sbr{\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}}}\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \adjt{\prn{\adjt{\mtrxof{I,f,e}}}}\adjt{\mtrxofb{\sqrttat,f}}\adjt{\mtrxof{S,f,e}}\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\adjt{\prn{\sqrttat}},f}\mtrxof{\adjt{S},e,f}\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{S}S,f,f}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\mtrxof{I,f,f}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{I},e,f}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}\mtrxof{I,e,f}\mtrxof{f_j,e} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \pmtrx{\innprd{e_1}{f_1}&\dotsb&\innprd{e_n}{f_1}\\\vdots&\ddots&\vdots\\\innprd{e_1}{f_n}&\dotsb&\innprd{e_n}{f_n}} \pmtrx{\innprd{f_j}{e_1}\\\vdots\\\innprd{f_j}{e_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \pmtrx{\sum_{k=1}^n\innprd{e_k}{f_1}\innprd{f_j}{e_k}\\\vdots\\\sum_{k=1}^n\innprd{e_k}{f_n}\innprd{f_j}{e_k}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \pmtrx{\sum_{k=1}^n\innprdbg{\innprd{f_j}{e_k}e_k}{f_1}\\\vdots\\\sum_{k=1}^n\innprdbg{\innprd{f_j}{e_k}e_k}{f_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \pmtrx{\innprdBg{\sum_{k=1}^n\innprd{f_j}{e_k}e_k}{f_1}\\\vdots\\\innprdBg{\sum_{k=1}^n\innprd{f_j}{e_k}e_k}{f_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \pmtrx{\innprd{f_j}{f_1}\\\vdots\\\innprd{f_j}{f_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \epsilon_j \\\\ &= \pmtrx{\lambda_1^2\innprd{f_1}{e_1}&\dotsb&\lambda_n^2\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\lambda_1^2\innprd{f_1}{e_n}&\dotsb&\lambda_n^2\innprd{f_n}{e_n}} \epsilon_j \\\\ &= \pmtrx{\lambda_j^2\innprd{f_j}{e_1}\\\vdots\\\lambda_j^2\innprd{f_j}{e_n}} \\\\ &= \lambda_j^2\pmtrx{\innprd{f_j}{e_1}\\\vdots\\\innprd{f_j}{e_n}} \\\\ &= \lambda_j^2\mtrxof{f_j,e} }$$
Proof of (F) Let $\epsilon_1,\dots,\epsilon_n$ denote the standard basis in $\wR^n$. Then
$$\align{ \mtrxof{\tta,e}\mtrxof{Sf_j,e} &= \mtrxof{T,e}\mtrxof{\adjtT,e}\mtrxof{Sf_j,e} \\\\ &= \mtrxof{T,e}\adjt{\mtrxof{T,e}}\mtrxof{Sf_j,e} \\\\ &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\adjt{\sbr{\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}}}\mtrxof{Sf_j,e} \\\\ &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{I},e,f}\adjt{\prn{\adjt{\mtrxof{I,f,e}}}}\adjt{\mtrxofb{\sqrttat,f}}\adjt{\mtrxof{S,f,e}}\mtrxof{Sf_j,e} \\\\ &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\mtrxof{I,e,f}\mtrxof{I,f,e}\mtrxofb{\adjt{\prn{\sqrttat}},f}\mtrxof{\adjt{S},e,f}\mtrxof{Sf_j,e} \\\\ &= \pmtrx{\mtrxof{S,f,e}_{:,1}&\dotsb&\mtrxof{S,f,e}_{:,n}}\mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{S}Sf_j,f} \\\\ &= \pmtrx{\mtrxof{Sf_1,e}&\dotsb&\mtrxof{Sf_n,e}}\mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}\mtrxof{f_j,f} \\\\ &= \pmtrx{\innprd{Sf_1}{e_1}&\dotsb&\innprd{Sf_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{e_n}&\dotsb&\innprd{Sf_n}{e_n}} \pmtrx{\lambda_1^2&&\\&\ddots&\\&&\lambda_n^2} \pmtrx{\innprd{f_j}{f_1}\\\vdots\\\innprd{f_j}{f_n}} \\\\ &= \pmtrx{\lambda_1^2\innprd{Sf_1}{e_1}&\dotsb&\lambda_n^2\innprd{Sf_n}{e_1}\\\vdots&\ddots&\vdots\\\lambda_1^2\innprd{Sf_1}{e_n}&\dotsb&\lambda_n^2\innprd{Sf_n}{e_n}} \epsilon_j \\\\ &= \pmtrx{\lambda_j^2\innprd{Sf_j}{e_1}\\\vdots\\\lambda_j^2\innprd{Sf_j}{e_n}} \\\\ &= \lambda_j^2\pmtrx{\innprd{Sf_j}{e_1}\\\vdots\\\innprd{Sf_j}{e_n}} \\\\ &= \lambda_j^2\mtrxof{Sf_j,e} }$$
Proof of (G)
$$\align{ \mtrxof{T,e} &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} \\ &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{I},e,f} \\ &= \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\mtrxof{I,e,f} \\\\ &= \pmtrx{\innprd{Sf_1}{e_1}&\dotsb&\innprd{Sf_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{e_n}&\dotsb&\innprd{Sf_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\innprd{e_1}{f_1}&\dotsb&\innprd{e_n}{f_1}\\\vdots&\ddots&\vdots\\\innprd{e_1}{f_n}&\dotsb&\innprd{e_n}{f_n}} \\\\ &= \pmtrx{\innprd{Sf_1}{e_1}&\dotsb&\innprd{Sf_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{e_n}&\dotsb&\innprd{Sf_n}{e_n}} \pmtrx{\lambda_1\innprd{e_1}{f_1}&\dotsb&\lambda_1\innprd{e_n}{f_1}\\\vdots&\ddots&\vdots\\\lambda_n\innprd{e_1}{f_n}&\dotsb&\lambda_n\innprd{e_n}{f_n}} \\\\ &= \pmtrx{\sum_{j=1}^n\lambda_j\innprd{Sf_j}{e_1}\innprd{e_1}{f_j}&\dotsb&\sum_{j=1}^n\lambda_j\innprd{Sf_j}{e_1}\innprd{e_n}{f_j}\\\vdots&\ddots&\vdots\\\sum_{j=1}^n\lambda_j\innprd{Sf_j}{e_n}\innprd{e_1}{f_j}&\dotsb&\sum_{j=1}^n\lambda_j\innprd{Sf_j}{e_n}\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\pmtrx{\lambda_j\innprd{Sf_j}{e_1}\innprd{e_1}{f_j}&\dotsb&\lambda_j\innprd{Sf_j}{e_1}\innprd{e_n}{f_j}\\\vdots&\ddots&\vdots\\\lambda_j\innprd{Sf_j}{e_n}\innprd{e_1}{f_j}&\dotsb&\lambda_j\innprd{Sf_j}{e_n}\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\lambda_j\pmtrx{\innprd{Sf_j}{e_1}\innprd{e_1}{f_j}&\dotsb&\innprd{Sf_j}{e_1}\innprd{e_n}{f_j}\\\vdots&\ddots&\vdots\\\innprd{Sf_j}{e_n}\innprd{e_1}{f_j}&\dotsb&\innprd{Sf_j}{e_n}\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\lambda_j\pmtrx{\innprd{Sf_j}{e_1}\\\vdots\\\innprd{Sf_j}{e_n}} \pmtrx{\innprd{e_1}{f_j}&\dotsb&\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\lambda_j\mtrxof{Sf_j,e}\pmtrx{\cj{\innprd{f_j}{e_1}}&\dotsb&\cj{\innprd{f_j}{e_n}}} \\\\ &= \sum_{j=1}^n\lambda_j\mtrxof{Sf_j,e}\pmtrx{\cj{\innprd{f_j}{e_1}}\\\vdots\\\cj{\innprd{f_j}{e_n}}}^t \\\\ &= \sum_{j=1}^n\lambda_j\mtrxof{Sf_j,e}\adjt{\pmtrx{\innprd{f_j}{e_1}\\\vdots\\\innprd{f_j}{e_n}}} \\\\ &= \sum_{j=1}^n\lambda_j\mtrxof{Sf_j,e}\adjt{\mtrxof{f_j,e}} }$$
$\wes$
Example W.7.SVD.3 Find the SVD of
$$ \mtrxof{T,e}\equiv\pmtrx{5&5\\-1&7} $$
where $e\equiv e_1,e_2$ is the standard basis of $\wR^2$. Note that
$$\align{ &Te_1=5e_1-1e_2=(5,-1) \\ &Te_2=5e_1+7e_2=(5,7) }$$
or
$$\align{ T(x,y) = T(xe_1+ye_2) = xTe_1+yTe_2 = x(5,-1)+y(5,7) = (5x,-x)+(5y,7y) = (5x+5y,-x+7y) \tag{W.7.SVD.3.0} }$$
Hence
$$\align{ \tat e_1 &= \adjtT(Te_1) \\ &= \innprd{Te_1}{Te_1}e_1+\innprd{Te_1}{Te_2}e_2 \\ &= \innprd{(5,-1)}{(5,-1)}e_1+\innprd{(5,-1)}{(5,7)}e_2 \\ &= (25+1)e_1+(25-7)e_2 \\ &= 26e_1+18e_2 }$$
and
$$\align{ \tat e_2 &= \adjtT(Te_2) \\ &= \innprd{Te_2}{Te_1}e_1+\innprd{Te_2}{Te_2}e_2 \\ &= \innprd{(5,7)}{(5,-1)}e_1+\innprd{(5,7)}{(5,7)}e_2 \\ &= (25-7)e_1+(25+49)e_2 \\ &= 18e_1+74e_2 }$$
Hence
$$\align{ \tat(x,y) &= \tat(xe_1+ye_2) \\ &= x\tat e_1+y\tat e_2 \\ &= x(26e_1+18e_2)+y(18e_1+74e_2) \\ &= x26e_1+x18e_2+y18e_1+y74e_2 \\ &= x26e_1+y18e_1+x18e_2+y74e_2 \\ &= (x26+y18)e_1+(x18+y74)e_2 \\ &= (x26+y18,x18+y74) \\ &= (26x+18y,18x+74y) \tag{W.7.SVD.3.1} }$$
Let’s find the eigenvalues of $\tat$. Then we’ll find corresponding eigenvectors. Then we’ll apply Gram-Schmidt to get an orthonormal eigenbasis of $\wR^2$ corresponding to $\tat$. Actually, since $\tat$ is self-adjoint, 7.22 and W.6.G.17 imply that it’s not necessary to perform the full Gram-Schmidt; we just need to normalize each of the eigenvectors. Then we’ll use W.7.G.21 to get an orthonormal eigenbasis of $\wR^2$ corresponding to $\sqrttat$.
Note that 7.13 implies that $\tat$ has real eigenvalues. The eigenpair equation is
$$ (\lambda x,\lambda y)=\lambda(x,y)=\tat(x,y)=(26x+18y,18x+74y) $$
This becomes the system
$$\align{ &26x+18y=\lambda x \\ &18x+74y=\lambda y }$$
If $x=0$, then $18y=0$ and $y=0$. And if $y=0$, then $18x=0$ and $x=0$. Since eigenvectors cannot be zero, then it must be that $x\neq0$ and $y\neq0$. Also note that
$$ 18y=\lambda x-26x=(\lambda-26)x \dq\implies\dq y=\frac{\lambda-26}{18}x \tag{W.7.SVD.3.2} $$
and
$$ 18x=\lambda y-74y=(\lambda-74)y=(\lambda-74)\frac{\lambda-26}{18}x $$
or
$$ 18^2=(\lambda-74)(\lambda-26)=\lambda^2-26\lambda-74\lambda+74\cdot26 $$
or
$$ 0=\lambda^2-100\lambda+74\cdot26-18^2 $$
def quad_form(a=1,b=1,c=1):
disc = b**2-4*a*c
if disc < 0:
raise ValueError('The discriminant b^2-4ac={} is less than zero.'.format(disc))
p1 = -b/(2*a)
p2 = np.sqrt(disc)/(2*a)
return (p1+p2,p1-p2)
quad_form(a=1,b=-100,c=74*26-18**2)
(80.0, 20.0)
def compute_MTaTe(MTe):
MTae=MTe.conj().T
MTaTe=np.matmul(MTae,MTe)
return MTaTe
MTe=np.array([[5,5],[-1,7]])
MTaTe=compute_MTaTe(MTe)
MTaTe
array([[26, 18],
[18, 74]])
def real_eigns_2x2(npa=np.array([[1,0],[0,1]])):
b = -(npa[0,0]+npa[1,1])
c = npa[0,0]*npa[1,1]-npa[0,1]*npa[1,0]
eignvals = quad_form(a=1,b=b,c=c)
y0 = (eignvals[0]-npa[0,0])/npa[0,1] # by W.7.SVD.3.2
y1 = (eignvals[1]-npa[0,0])/npa[0,1] # by W.7.SVD.3.2
evctr0 = np.array([1,y0])
evctr1 = np.array([1,y1])
return (eignvals,evctr0,evctr1)
# by 7.13, MTaTe has real eigenvalues
eigvals,v1,v2=real_eigns_2x2(MTaTe)
eigvals
(80.0, 20.0)
v1
array([ 1., 3.])
v2
array([ 1. , -0.33333333])
Let’s verify these. From W.7.SVD.3.1, we have
$$\align{ &\tat(1,3) = (26\cdot1+18\cdot3,18\cdot1+74\cdot3) = (26+54,18+222) = (80,240) = 80(1,3) \\\\ &\tat\Prn{1,-\frac13} = \Prn{26\cdot1-18\cdot\frac13,18\cdot1-74\cdot\frac13} = \Prn{26-6,\frac{54-74}3} = \Prn{20,-\frac{20}3} = 20\Prn{1,-\frac13} }$$
Next we normalize this eigenbasis:
$$\align{ &f_1\equiv\frac{(1,3)}{\dnorm{(1,3)}} = \frac{(1,3)}{\sqrt{1+9}} = \frac1{\sqrt{10}}(1,3) \\\\ &f_2\equiv\frac{\prn{1,-\frac13}}{\dnorm{\prn{1,-\frac13}}} = \frac{\prn{1,-\frac13}}{\sqrt{1+\frac19}} = \frac{\prn{1,-\frac13}}{\sqrt{\frac{10}{9}}} = \frac{\prn{1,-\frac13}}{\frac{\sqrt{10}}3} = \frac{\prn{3,-1}}{\sqrt{10}} = \frac1{\sqrt{10}}(3,-1) }$$
or
def Rn_innprd(u,v):
if len(u) != len(v):
raise ValueError('Rn_innprd: vector lengths not equal: {} != {}'.format(len(u),len(v)))
return np.sum(u[i]*v[i] for i in range(len(u)))
Rn_norm=lambda v: np.sqrt(Rn_innprd(v,v))
def gram_schmidt(lin_ind_lst):
v0 = lin_ind_lst[0]
e0 = v0 / Rn_norm(v0)
ol = [e0]
for i in range(1,len(lin_ind_lst)):
vi = lin_ind_lst[i]
wi = vi - np.sum(Rn_innprd(vi,ol[j]) * ol[j] for j in range(i))
ei = wi / Rn_norm(wi)
ol.append(ei)
return ol
# quick check:
v = [np.array([41,41]),np.array([-17,0])]
gram_schmidt(v)
[array([ 0.70710678, 0.70710678]), array([-0.70710678, 0.70710678])]
v = [v1,v2]
f = gram_schmidt(v)
f
[array([ 0.31622777, 0.9486833 ]), array([ 0.9486833 , -0.31622777])]
def normalize(lst):
nl = []
for i in range(len(lst)):
vi = lst[i]
ni = vi / Rn_norm(vi)
nl.append(ni)
return nl
# normalized eigenbase agrees with gram-schmidt applied to this eigenbase:
normalize(v)
[array([ 0.31622777, 0.9486833 ]), array([ 0.9486833 , -0.31622777])]
# check against our manually computed f1 and f2:
mcf1=(1/np.sqrt(10))*np.array([1,3])
mcf2=(1/np.sqrt(10))*np.array([3,-1])
mcf1,mcf2
(array([ 0.31622777, 0.9486833 ]), array([ 0.9486833 , -0.31622777]))
By W.7.G.21, $\sqrttat$ is defined by
$$ \sqrttat f_1 = \sqrt{80}f_1 \dq\dq \sqrttat f_2 = \sqrt{20}f_2 $$
Define $f\equiv f_1,f_2$. Then $\mtrxofb{\sqrttat,f}$ is diagonal and the eigenvalues of $\sqrttat$ are precisely the entries on the diagonal:
$$ \mtrxofb{\sqrttat,f}=\pmtrx{\sqrt{80}&0\\0&\sqrt{20}} $$
W.7.G.41 implies that the list $\sqrttat f_1,\sqrttat f_2$ is orthogonal. Hence $\sqrttat f_1,\sqrttat f_2$ is linearly independent and hence is a basis of $\rangspb{\sqrttat}=\wR^2$.
Now define $S\in\oper{\wR^2}$ by
$$ S\prn{\sqrttat f_1} \equiv Tf_1 \dq\dq S\prn{\sqrttat f_2} \equiv Tf_2 \tag{W.7.SVD.3.3} $$
7.46 and 7.47 imply that $S$ is an isometry. Note that
$$\align{ \sqrt{80}Sf_1 &= S\prn{\sqrt{80}f_1} \\ &= S\prn{\sqrttat f_1} \\ &= Tf_1 \\ &= T\Prn{\frac1{\sqrt{10}}(1,3)} \\ &= \frac1{\sqrt{10}}T(1,3) \\ &= \frac1{\sqrt{10}}(5\cdot1+5\cdot3,-1+7\cdot3) \\ &= \frac1{\sqrt{10}}(20,20) }$$
and
$$ Sf_1 = \frac1{\sqrt{80}}\frac1{\sqrt{10}}(20,20) = \frac1{\sqrt{800}}\prn{\sqrt{400},\sqrt{400}}=\Prn{\frac1{\sqrt2},\frac1{\sqrt2}} = \frac1{\sqrt2}e_1+\frac1{\sqrt2}e_2 $$
Similarly
$$\align{ \sqrt{20}Sf_2 &= S\prn{\sqrt{20}f_2} \\ &= S\prn{\sqrttat f_2} \\ &= Tf_2 \\ &= T\Prn{\frac1{\sqrt{10}}(3,-1)} \\ &= \frac1{\sqrt{10}}T(3,-1) \\ &= \frac1{\sqrt{10}}(5\cdot3-5\cdot1,-3-7\cdot1) \\ &= \frac1{\sqrt{10}}(10,-10) }$$
and
$$ Sf_2 = \frac1{\sqrt{20}}\frac1{\sqrt{10}}(10,-10) = \frac1{\sqrt{200}}\prn{\sqrt{100},-\sqrt{100}}=\Prn{\frac1{\sqrt2},-\frac1{\sqrt2}} = \frac1{\sqrt2}e_1-\frac1{\sqrt2}e_2 $$
and
$$\align{ \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} &= \pmtrx{\frac1{\sqrt2}&\frac1{\sqrt2}\\\frac1{\sqrt2}&-\frac1{\sqrt2}}\pmtrx{\sqrt{80}&0\\0&\sqrt{20}}\adjt{\pmtrx{\frac{1}{\sqrt{10}}&\frac{3}{\sqrt{10}}\\\frac{3}{\sqrt{10}}&-\frac{1}{\sqrt{10}}}} \\\\ &= \pmtrx{\frac{\sqrt{80}}{\sqrt{2}}&\frac{\sqrt{20}}{\sqrt{2}}\\\frac{\sqrt{80}}{\sqrt{2}}&-\frac{\sqrt{20}}{\sqrt{2}}}\pmtrx{\frac{1}{\sqrt{10}}&\frac{3}{\sqrt{10}}\\\frac{3}{\sqrt{10}}&-\frac{1}{\sqrt{10}}} \\\\ &= \pmtrx{\frac{\sqrt{80}}{\sqrt{2}\sqrt{10}}+\frac{3\sqrt{20}}{\sqrt{2}\sqrt{10}} & \frac{3\sqrt{80}}{\sqrt{2}\sqrt{10}}-\frac{\sqrt{20}}{\sqrt{2}\sqrt{10}} \\ \frac{\sqrt{80}}{\sqrt{2}\sqrt{10}}-\frac{3\sqrt{20}}{\sqrt{2}\sqrt{10}} & \frac{3\sqrt{80}}{\sqrt{2}\sqrt{10}}+\frac{\sqrt{20}}{\sqrt{2}\sqrt{10}} } \\\\ &= \pmtrx{\sqrt{\frac{80}{20}}+3\sqrt{\frac{20}{20}}&3\sqrt{\frac{80}{20}}-\sqrt{\frac{20}{20}} \\ \sqrt{\frac{80}{20}}-3\sqrt{\frac{20}{20}}&3\sqrt{\frac{80}{20}}+\sqrt{\frac{20}{20}} } \\\\ &= \pmtrx{2+3&3\cdot2-1\\2-3&3\cdot2+1} \\\\ &= \pmtrx{5&5\\-1&7} \\\\ &= \mtrxof{T,e} }$$
In W.7.SVD.3.3, we defined $S\in\oper{\wR^2}$ by
$$ S\prn{\sqrttat f_1} \equiv Tf_1 \dq\dq S\prn{\sqrttat f_2} \equiv Tf_2 $$
But W.7.PLRDC.4 gives an equivalent and slightly more efficient definition:
$$ Sf_1\equiv\frac1{\sqrt{\lambda_1}}Tf_1 \dq\dq Sf_2\equiv\frac1{\sqrt{\lambda_2}}Tf_2 $$
Then
$$\align{ Sf_1 &= \frac1{\sqrt{80}}Tf_1 \\ &= \frac1{\sqrt{80}}T\Prn{\frac1{\sqrt{10}}(1,3)} \\ &= \frac1{\sqrt{80}}\frac1{\sqrt{10}}T(1,3) \\ &= \frac1{\sqrt{800}}(5\cdot1+5\cdot3,-1+7\cdot3) \\ &= \frac1{\sqrt{800}}(20,20) \\ &= \frac1{\sqrt{800}}\prn{\sqrt{400},\sqrt{400}} \\ &= \Prn{\frac1{\sqrt2},\frac1{\sqrt2}} \\ &= \frac1{\sqrt2}e_1+\frac1{\sqrt2}e_2 }$$
and similarly for $Sf_2$.
$\wes$
Example W.7.SVD.4 Find the SVD of
$$ \mtrxof{T,e}\equiv\pmtrx{5&5\\-1&-1} $$
where $e\equiv e_1,e_2$ is the standard basis of $\wR^2$. Note that
MTe=np.array([[5,5],[-1,-1]])
MTaTe=compute_MTaTe(MTe)
MTaTe
array([[26, 26],
[26, 26]])
eigvals,v1,v2=real_eigns_2x2(MTaTe)
eigvals
(52.0, 0.0)
v1
array([ 1., 1.])
v2
array([ 1., -1.])
Next we normalize this eigenbasis:
$$\align{ &f_1\equiv\frac{(1,1)}{\dnorm{(1,1)}} = \frac{(1,1)}{\sqrt{1+1}} = \frac1{\sqrt{2}}(1,1) \\\\ &f_2\equiv\frac{(1,-1)}{\dnorm{(1,-1)}} = \frac{(1,-1)}{\sqrt{1+1}} = \frac1{\sqrt{2}}(1,-1) }$$
or
v = [v1,v2]
f1,f2=normalize(v)
f1
array([ 0.70710678, 0.70710678])
f2
array([ 0.70710678, -0.70710678])
Note that W.7.PLRDC.3(A) implies that $f_1$ is an orthonormal basis of $\rangspb{\sqrttat}$ because the eigenvalue corresponding to $f_1$ is positive. And W.7.PLRDC.3(B) implies that $f_2$ is an orthonormal basis of $\rangspb{\sqrttat}^\perp$ because the eigenvalue corresponding to $f_2$ is zero.
Now define $S_1\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ by
$$\align{ S_1f_1 &\equiv \frac1{\sqrt{\lambda_1}}Tf_1 \\ &= \frac1{\sqrt{52}}T\Prn{\frac1{\sqrt2}(1,1)} \\ &= \frac1{\sqrt{52}}\frac1{\sqrt2}T(1,1) \\ &= \frac1{\sqrt{104}}\prn{5\cdot1+5\cdot1,(-1)\cdot1+(-1)\cdot1} \\ &= \frac1{\sqrt{104}}\prn{5+5,(-1)+(-1)} \\ &= \frac1{\sqrt{104}}(10,-2) }$$
And define $S_2\in\linmapb{\rangspb{\sqrttat}^\perp}{\rangsp{T}^\perp}$ by
$$ S_2f_2\equiv\frac1{\sqrt{104}}(2,10) $$
Note that 6.47 gives $\wR^2=\rangspb{\sqrttat}\oplus\rangspb{\sqrttat}^\perp$. Hence any $v\in\wR^2$ can be uniquely decomposed by $v=u+w$ where $u\in\rangspb{\sqrttat}$ and $w\in\rangspb{\sqrttat}^\perp$. So we can define $S\in\oper{\wR^2}$ by
$$\align{ Sv\equiv S_1u+S_2w }$$
Since $f_1\in\rangspb{\sqrttat}$, its unique decomposition is $f_1=f_1+0$ where $0\in\rangspb{\sqrttat}^\perp$. Hence $Sf_1=S_1f_1+S_20=S_1f_1$. And similarly, $Sf_2=S_2f_2$. Hence
$$\align{ \mtrxof{S,f,e} &= \pmtrx{\mtrxof{S,f,e}_{:,1}&\mtrxof{S,f,e}_{:,2}} \\ &= \pmtrx{\mtrxof{Sf_1,e}&\mtrxof{Sf_2,e}} \\ &= \pmtrx{\frac{10}{\sqrt{104}}&\frac{2}{\sqrt{104}}\\-\frac{2}{\sqrt{104}}&\frac{10}{\sqrt{104}}} }$$
and
$$\align{ \mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} &= \pmtrx{\frac{10}{\sqrt{104}}&\frac{2}{\sqrt{104}}\\-\frac{2}{\sqrt{104}}&\frac{10}{\sqrt{104}}}\pmtrx{\sqrt{52}&0\\0&0}\adjt{\pmtrx{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}}} \\\\ &= \pmtrx{\frac{10\sqrt{52}}{\sqrt{104}}&0\\-\frac{2\sqrt{52}}{\sqrt{104}}&0}\pmtrx{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}} \\\\ &= \pmtrx{\frac{10}{\sqrt{2}}&0\\-\frac{2}{\sqrt{2}}&0}\pmtrx{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}} \\\\ &= \pmtrx{\frac{10}{2}&\frac{10}{2}\\-\frac22&-\frac22} \\\\ &= \pmtrx{5&5\\-1&-1} \\\\ &= \mtrxof{T,e} }$$
Proposition W.7.SVD.5 Let $T\in\linmap{V}{W}$, let $e\equiv e_1,\dots,e_n$ denote any orthonormal basis of $V$, and let $g\equiv g_1,\dots,g_m$ denote an orthonormal basis of $W$.
If $\dim{W}\geq\dim{V}$, then (A)-(G) hold:
(A) There exist a subspace $U$ of $W$ with $\dim{U}=\dim{V}$, an isometry $S\in\linmap{V}{U}$, and an orthonormal basis $f$ of $V$ such that
$$ \mtrxof{T,e,g}=\mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} \tag{A} $$
where $\mtrxofb{\sqrttat,f}$ is diagonal and
$$ \mtrxofb{\sqrttat,f}^2=\pmtrx{\sqrt{\lambda_1}&&\\&\ddots&\\&&\sqrt{\lambda_n}}^2 =\pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n}=\mtrxof{\tat,f} $$
(B) The square roots of the eigenvalues of $\tat$ are precisely the diagonal entries of $\mtrxofb{\sqrttat,f}$, with each eigenvalue $\lambda_j$ repeated $\dim{\eignsp{\lambda_j}{\tat}}$ times.
(C) The columns of $\mtrxof{S,f,g}$ and $\mtrxof{I,f,e}$ are orthonormal with respect to $\innprddt_{\wF^m}$ and $\innprddt_{\wF^n}$.
(D) The singular values of $T$ are precisely the square roots of the diagonal entries of $\mtrxof{\tat,f}$, with each singular value $\sqrt{\lambda_j}$ repeated $\dim{\eignsp{\lambda_j}{\tat}}$ times.
(E) The columns of $\mtrxof{I,f,e}$ are eigenvectors of $\mtrxof{\tat,e}$. That is, $\mtrxof{f_j,e}$ is an eigenvector of $\mtrxof{\tat,e}$ corresponding to eigenvalue $\lambda_j$.
(F) The columns of $\mtrxof{S,f,g}$ are eigenvectors of $\mtrxof{\tta,g}$. That is, $\mtrxof{Sf_j,g}$ is an eigenvector of $\mtrxof{\tta,g}$ corresponding to eigenvalue $\lambda_j$.
(G) $\mtrxof{T,e,g}=\sum_{j=1}^{\dim{\rangsp{T}}}\sqrt{\lambda_j}\mtrxof{Sf_j,g}\adjt{\mtrxof{f_j,e}}$
(H) If $\dim{W}\leq\dim{V}$, then there exist a subspace $U$ of $V$ with $\dim{U}=\dim{W}$, an isometry $S\in\linmap{U}{W}$, and an orthonormal basis $f$ of $W$ such that
$$ \mtrxof{T,e,g} = \mtrxof{I,f,g}\mtrxofb{\sqrttta,f}\adjt{\mtrxof{\adjt{S},f,e}} \tag{H} $$
Proof of (A) Note that $\tat$ is a positive operator:
$$ \adjt{(\tat)}=\adjtT\adjt{(\adjtT)}=\tat $$
And for any $v\in V$:
$$ \innprd{\tat v}{v}_V=\innprd{Tv}{\adjt{(\adjtT)}v}_W=\innprd{Tv}{Tv}_W=\dnorm{Tv}_W^2\geq0 $$
Then the Spectral Theorem (W.7.SVD.1) gives an orthonormal basis $f\equiv f_1,\dots,f_n$ of $V$ consisting of eigenvectors of $\tat$ corresponding to eigenvalues $\lambda_1,\dots,\lambda_n$ so that $\mtrxof{\tat,f}$ is diagonal and
$$ \mtrxof{\tat,e}=\mtrxof{I,f,e}\mtrxof{\tat,f}\adjt{\mtrxof{I,f,e}} $$
Then W.7.G.21 gives that $f_1,\dots,f_n$ consists of eigenvectors of $\sqrttat$ corresponding to eigenvalues $\sqrt{\lambda_1},\dots,\sqrt{\lambda_n}$ so that $\mtrxofb{\sqrttat,f}$ is diagonal and
$$ \mtrxofb{\sqrttat,e}=\mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} $$
Let $S\sqrttat$ be the Polar Decomposition of $T$ (W.7.PLRDC.5). Then $S\sqrttat v=Tv$ for all $v\in V$. In particular, $S\sqrttat e_j=Te_j$ for $j=1,\dots,n$. Hence
$$\align{ \mtrxof{T,e,g} &= \mtrxofb{S\sqrttat,e,g}\tag{by W.7.PLRDC.5} \\ &= \mtrxof{S,e,g}\mtrxofb{\sqrttat,e,e}\tag{by W.3.27} \\ &= \mtrxof{SI,e,g}\mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\tag{by W.7.SVD.1} \\ &= \mtrxof{S,f,g}\mtrxof{I,e,f}\mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\tag{by W.3.27} \\ &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\tag{by W.3.28} }$$
Proof of (B) 5.32 implies that the eigenvalues of $\tat$ are precisely the diagonal entries of $\mtrxof{\tat,f}$. And exercise 5.C.7 implies that each eigenvalue $\lambda_j$ is repeated $\dim{\eignsp{\lambda_j}{\tat}}$ times on the diagonal. And W.7.G.21 shows that the square roots of these eigenvalues are precisely the diagonal entries of $\mtrxofb{\sqrttat,f}$.
Proof of (C) Note that $Sf_1,\dots,Sf_n$ is orthonormal (by W.7.G.43) since $S$ is an isometry. Also note that 6.30 gives
$$ Sf_k=\sum_{i=1}^m\innprd{Sf_k}{g_i}_Wg_i $$
Then
$$\align{ \mtrxof{S,f,g} &= \pmtrx{\mtrxof{S,f,g}_{:,1}&\dotsb&\mtrxof{S,f,g}_{:,n}} \\ &= \pmtrx{\mtrxof{Sf_1,g}&\dotsb&\mtrxof{Sf_n,g}} \\ &= \pmtrx{\innprd{Sf_1}{g_1}_W&\dotsb&\innprd{Sf_n}{g_1}_W\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{g_m}_W&\dotsb&\innprd{Sf_n}{g_m}_W} \\ }$$
and
$$\align{ \innprdbg{\mtrxof{S,f,g}_{:,j}}{\mtrxof{S,f,g}_{:,k}}_{\wF^m} &= \sum_{i=1}^m\innprd{Sf_j}{g_i}_W\cj{\innprd{Sf_k}{g_i}_W}\tag{by W.6.G.9} \\ &= \sum_{i=1}^m\innprdbg{Sf_j}{\innprd{Sf_k}{g_i}_Wg_i}_W \\ &= \innprdBgg{Sf_j}{\sum_{i=1}^m\innprd{Sf_k}{g_i}_Wg_i}_W \\ &= \innprd{Sf_j}{Sf_k}_W\tag{by 6.30} \\ &= \cases{1&j=k\\0&j\neq k} }$$
Hence the columns of $\mtrxof{S,f,g}$ are orthonormal with respect to $\innprddt_{\wF^m}$. Since $If_1,\dots,If_n$ is orthonormal, the proof is similar that the columns of $\mtrxof{I,f,e}$ are orthonormal with respect to $\innprddt_{\wF^n}$.
Proof of (D) This follows from W.7.G.22.
Proof of (E) Note that $f_j$ was chosen to be an eigenvector of $\tat$ corresponding to eigenvalue $\lambda_j$. Hence $\tat f_j=\lambda_jf_j$ and
$$ \mtrxof{\tat,e}\mtrxof{f_j,e}=\mtrxof{\tat f_j,e}=\mtrxof{\lambda_jf_j,e}=\lambda_j\mtrxof{f_j,e} $$
Alternatively, note that
$$ \mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}=\pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} $$
And let $\epsilon_1,\dots,\epsilon_n$ denote the standard basis in $\wR^n$. Then
$$\align{ \mtrxof{\tat,e}\mtrxof{f_j,e} &= \mtrxof{\adjtT,g,e}\mtrxof{T,e,g}\mtrxof{f_j,e} \\\\ &= \adjt{\mtrxof{T,e,g}}\mtrxof{T,e,g}\mtrxof{f_j,e} \\\\ &= \adjt{\sbr{\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}}}\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \adjt{\prn{\adjt{\mtrxof{I,f,e}}}}\adjt{\mtrxofb{\sqrttat,f}}\adjt{\mtrxof{S,f,e}}\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\adjt{\prn{\sqrttat}},f}\mtrxof{\adjt{S},e,f}\mtrxof{S,f,e}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{S}S,f,f}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\mtrxof{I,f,f}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{I},e,f}\mtrxof{f_j,e} \\\\ &= \mtrxof{I,f,e}\mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}\mtrxof{I,e,f}\mtrxof{f_j,e} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\innprd{e_1}{f_1}&\dotsb&\innprd{e_n}{f_1}\\\vdots&\ddots&\vdots\\\innprd{e_1}{f_n}&\dotsb&\innprd{e_n}{f_n}} \pmtrx{\innprd{f_j}{e_1}\\\vdots\\\innprd{f_j}{e_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\sum_{k=1}^n\innprd{e_k}{f_1}\innprd{f_j}{e_k}\\\vdots\\\sum_{k=1}^n\innprd{e_k}{f_n}\innprd{f_j}{e_k}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\sum_{k=1}^n\innprdbg{\innprd{f_j}{e_k}e_k}{f_1}\\\vdots\\\sum_{k=1}^n\innprdbg{\innprd{f_j}{e_k}e_k}{f_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\innprdBg{\sum_{k=1}^n\innprd{f_j}{e_k}e_k}{f_1}\\\vdots\\\innprdBg{\sum_{k=1}^n\innprd{f_j}{e_k}e_k}{f_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\innprd{f_j}{f_1}\\\vdots\\\innprd{f_j}{f_n}} \\\\ &= \pmtrx{\innprd{f_1}{e_1}&\dotsb&\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\innprd{f_1}{e_n}&\dotsb&\innprd{f_n}{e_n}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \epsilon_j \\\\ &= \pmtrx{\lambda_1\innprd{f_1}{e_1}&\dotsb&\lambda_n\innprd{f_n}{e_1}\\\vdots&\ddots&\vdots\\\lambda_1\innprd{f_1}{e_n}&\dotsb&\lambda_n\innprd{f_n}{e_n}} \epsilon_j \\\\ &= \pmtrx{\lambda_j\innprd{f_j}{e_1}\\\vdots\\\lambda_j\innprd{f_j}{e_n}} \\\\ &= \lambda_j\pmtrx{\innprd{f_j}{e_1}\\\vdots\\\innprd{f_j}{e_n}} \\\\ &= \lambda_j\mtrxof{f_j,e} }$$
Proof of (F) By W.7.PLRDC.5.2, for $j=1,\dots,r$, we have
$$ \tta Sf_j=\tta\frac1{\sqrt{\lambda_j}}Tf_j=\frac1{\sqrt{\lambda_j}}\tta Tf_j=\frac1{\sqrt{\lambda_j}}T(\tat f_j)=\frac1{\sqrt{\lambda_j}}T(\lambda_jf_j)=\lambda_j\frac1{\sqrt{\lambda_j}}Tf_j=\lambda_jSf_j $$
and, for $j=r+1,\dots,n$, we have
$$ \tta Sf_j=\tta h_j=T(0)=0=0Sf_j=\lambda_jSf_j $$
The second equality follows from W.7.PLRDC.3(G). Hence
$$ \mtrxof{\tta,g}\mtrxof{Sf_j,g}=\mtrxof{\tta Sf_j,g}=\mtrxof{\lambda_jSf_j,g}=\lambda_j\mtrxof{Sf_j,g} $$
Alternatively, let $\epsilon_1,\dots,\epsilon_n$ denote the standard basis in $\wR^n$. Then
$$\align{ \mtrxof{\tta,g}\mtrxof{Sf_j,g} &= \mtrxof{T,e,g}\mtrxof{\adjtT,g,e}\mtrxof{Sf_j,g} \\\\ &= \mtrxof{T,e,g}\adjt{\mtrxof{T,e,g}}\mtrxof{Sf_j,g} \\\\ &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}\adjt{\sbr{\mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}}}}\mtrxof{Sf_j,g} \\\\ &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{I},e,f}\adjt{\prn{\adjt{\mtrxof{I,f,e}}}}\adjt{\mtrxofb{\sqrttat,f}}\adjt{\mtrxof{S,f,g}}\mtrxof{Sf_j,g} \\\\ &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\mtrxof{I,e,f}\mtrxof{I,f,e}\mtrxofb{\adjt{\prn{\sqrttat}},f}\mtrxof{\adjt{S},g,f}\mtrxof{Sf_j,g} \\\\ &= \pmtrx{\mtrxof{S,f,g}_{:,1}&\dotsb&\mtrxof{S,f,g}_{:,n}}\mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{S}Sf_j,f} \\\\ &= \pmtrx{\mtrxof{Sf_1,g}&\dotsb&\mtrxof{Sf_n,g}}\mtrxofb{\sqrttat,f}\mtrxofb{\sqrttat,f}\mtrxof{f_j,f} \\\\ &= \pmtrx{\innprd{Sf_1}{g_1}&\dotsb&\innprd{Sf_n}{g_1}\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{g_m}&\dotsb&\innprd{Sf_n}{g_m}} \pmtrx{\lambda_1&&\\&\ddots&\\&&\lambda_n} \pmtrx{\innprd{f_j}{f_1}\\\vdots\\\innprd{f_j}{f_n}} \\\\ &= \pmtrx{\lambda_1\innprd{Sf_1}{g_1}&\dotsb&\lambda_n\innprd{Sf_n}{g_1}\\\vdots&\ddots&\vdots\\\lambda_1\innprd{Sf_1}{g_m}&\dotsb&\lambda_n\innprd{Sf_n}{g_m}} \epsilon_j \\\\ &= \pmtrx{\lambda_j\innprd{Sf_j}{g_1}\\\vdots\\\lambda_j\innprd{Sf_j}{g_m}} \\\\ &= \lambda_j\pmtrx{\innprd{Sf_j}{g_1}\\\vdots\\\innprd{Sf_j}{g_m}} \\\\ &= \lambda_j\mtrxof{Sf_j,g} }$$
Proof of (G)
$$\align{ \mtrxof{T,e,g} &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} \\ &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\mtrxof{\adjt{I},e,f} \\ &= \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\mtrxof{I,e,f} \\\\ &= \pmtrx{\innprd{Sf_1}{g_1}&\dotsb&\innprd{Sf_n}{g_1}\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{g_m}&\dotsb&\innprd{Sf_n}{g_m}} \pmtrx{\sqrt{\lambda_1}&&\\&\ddots&\\&&\sqrt{\lambda_n}} \pmtrx{\innprd{e_1}{f_1}&\dotsb&\innprd{e_n}{f_1}\\\vdots&\ddots&\vdots\\\innprd{e_1}{f_n}&\dotsb&\innprd{e_n}{f_n}} \\\\ &= \pmtrx{\innprd{Sf_1}{g_1}&\dotsb&\innprd{Sf_n}{g_1}\\\vdots&\ddots&\vdots\\\innprd{Sf_1}{g_m}&\dotsb&\innprd{Sf_n}{g_m}} \pmtrx{\sqrt{\lambda_1}\innprd{e_1}{f_1}&\dotsb&\sqrt{\lambda_1}\innprd{e_n}{f_1}\\\vdots&\ddots&\vdots\\\sqrt{\lambda_n}\innprd{e_1}{f_n}&\dotsb&\sqrt{\lambda_n}\innprd{e_n}{f_n}} \\\\ &= \pmtrx{\sum_{j=1}^n\sqrt{\lambda_j}\innprd{Sf_j}{g_1}\innprd{e_1}{f_j}&\dotsb&\sum_{j=1}^n\sqrt{\lambda_j}\innprd{Sf_j}{g_1}\innprd{e_n}{f_j}\\\vdots&\ddots&\vdots\\\sum_{j=1}^n\sqrt{\lambda_j}\innprd{Sf_j}{g_m}\innprd{e_1}{f_j}&\dotsb&\sum_{j=1}^n\sqrt{\lambda_j}\innprd{Sf_j}{g_m}\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\pmtrx{\sqrt{\lambda_j}\innprd{Sf_j}{g_1}\innprd{e_1}{f_j}&\dotsb&\sqrt{\lambda_j}\innprd{Sf_j}{g_1}\innprd{e_n}{f_j}\\\vdots&\ddots&\vdots\\\sqrt{\lambda_j}\innprd{Sf_j}{g_m}\innprd{e_1}{f_j}&\dotsb&\sqrt{\lambda_j}\innprd{Sf_j}{g_m}\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\sqrt{\lambda_j}\pmtrx{\innprd{Sf_j}{g_1}\innprd{e_1}{f_j}&\dotsb&\innprd{Sf_j}{g_1}\innprd{e_n}{f_j}\\\vdots&\ddots&\vdots\\\innprd{Sf_j}{g_m}\innprd{e_1}{f_j}&\dotsb&\innprd{Sf_j}{g_m}\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\sqrt{\lambda_j}\pmtrx{\innprd{Sf_j}{g_1}\\\vdots\\\innprd{Sf_j}{g_m}} \pmtrx{\innprd{e_1}{f_j}&\dotsb&\innprd{e_n}{f_j}} \\\\ &= \sum_{j=1}^n\sqrt{\lambda_j}\mtrxof{Sf_j,g}\pmtrx{\cj{\innprd{f_j}{e_1}}&\dotsb&\cj{\innprd{f_j}{e_n}}} \\\\ &= \sum_{j=1}^n\sqrt{\lambda_j}\mtrxof{Sf_j,g}\pmtrx{\cj{\innprd{f_j}{e_1}}\\\vdots\\\cj{\innprd{f_j}{e_n}}}^t \\\\ &= \sum_{j=1}^n\sqrt{\lambda_j}\mtrxof{Sf_j,g}\adjt{\pmtrx{\innprd{f_j}{e_1}\\\vdots\\\innprd{f_j}{e_n}}} \\\\ &= \sum_{j=1}^n\sqrt{\lambda_j}\mtrxof{Sf_j,g}\adjt{\mtrxof{f_j,e}} \\ &= \sum_{j=1}^{\dim{\rangsp{T}}}\sqrt{\lambda_j}\mtrxof{Sf_j,g}\adjt{\mtrxof{f_j,e}} }$$
where the last equality follows because W.7.PLRDC.3(G) gives $\lambda_j=0$ for $j=\dim{\rangsp{T}}+1,\dots,n$.
Proof of (H) Note that $\tta$ is a positive operator:
$$ \adjt{(\tta)}=\adjt{(\adjtT)}\adjtT=\tta $$
And for any $w\in W$:
$$ \innprd{\tta w}{w}_W=\innprd{\adjtT w}{\adjtT w}_V=\dnorm{\adjtT w}_V^2\geq0 $$
Hence, by 7.36 and 7.44, $\sqrttta$ is the unique positive square root of $\tta$. Since $\sqrttta$ is positive, it’s also self-adjoint. Then W.7.SVD.1 gives an orthonormal basis $f$ of $W$ such that $\mtrxofb{\sqrttta,f}$ is diagonal and
$$ \mtrxofb{\sqrttta,g}=\mtrxof{I,f,g}\mtrxofb{\sqrttta,f}\adjt{\mtrxof{I,f,g}} $$
Hence
$$\align{ \mtrxof{T,e,g} &= \mtrxofb{\sqrttta S,e,g}\tag{by Polar Decomposition W.7.PLRDC.5} \\ &= \mtrxofb{\sqrttta,g,g}\mtrxof{S,e,g}\tag{by W.3.27} \\ &= \mtrxof{I,f,g}\mtrxofb{\sqrttta,f}\adjt{\mtrxof{I,f,g}}\mtrxof{IS,e,g}\tag{by W.7.SVD.1} \\ &= \mtrxof{I,f,g}\mtrxofb{\sqrttta,f}\mtrxof{\adjt{I},g,f}\mtrxof{I,f,g}\mtrxof{S,e,f}\tag{by W.3.27} \\ &= \mtrxof{I,f,g}\mtrxofb{\sqrttta,f}\mtrxof{I,g,f}\mtrxof{I,f,g}\adjt{\mtrxof{\adjt{S},f,e}}\tag{by W.7.G.35} \\ &= \mtrxof{I,f,g}\mtrxofb{\sqrttta,f}\adjt{\mtrxof{\adjt{S},f,e}}\tag{by W.3.28} }$$
$\wes$
Example W.7.SVD.6 Find the SVD of
$$ \mtrxof{T,e,g}\equiv\pmtrx{5&5\\-1&-1\\3&3} $$
where $e\equiv e_1,e_2$ is the standard basis of $\wR^2$ and where $g\equiv g_1,g_2,g_3$ is the standard basis of $\wR^3$. Note that
MTe=np.array([[5,5],[-1,-1],[3,3]])
MTe
array([[ 5, 5],
[-1, -1],
[ 3, 3]])
MTaTe=compute_MTaTe(MTe)
MTaTe
array([[35, 35],
[35, 35]])
eigvals,v1,v2=real_eigns_2x2(MTaTe)
eigvals
(70.0, 0.0)
v1
array([ 1., 1.])
v2
array([ 1., -1.])
Next we normalize this eigenbasis:
$$\align{ &f_1\equiv\frac{(1,1)}{\dnorm{(1,1)}} = \frac{(1,1)}{\sqrt{1+1}} = \frac1{\sqrt{2}}(1,1) \\\\ &f_2\equiv\frac{(1,-1)}{\dnorm{(1,-1)}} = \frac{(1,-1)}{\sqrt{1+1}} = \frac1{\sqrt{2}}(1,-1) }$$
or
v = [v1,v2]
f1,f2=normalize(v)
f1
array([ 0.70710678, 0.70710678])
f2
array([ 0.70710678, -0.70710678])
Note that W.7.PLRDC.3(A) implies that $f_1$ is an orthonormal basis of $\rangspb{\sqrttat}$ because the eigenvalue corresponding to $f_1$ is positive. And W.7.PLRDC.3(B) implies that $f_2$ is an orthonormal basis of $\rangspb{\sqrttat}^\perp$ because the eigenvalue corresponding to $f_2$ is zero.
Now define $S_1\in\linmapb{\rangspb{\sqrttat}}{\rangsp{T}}$ by
$$\align{ S_1f_1 &\equiv \frac1{\sqrt{\lambda_1}}Tf_1 \\ &= \frac1{\sqrt{70}}T\Prn{\frac1{\sqrt2}(1,1)} \\ &= \frac1{\sqrt{70}}\frac1{\sqrt2}T(1,1) \\ &= \frac1{\sqrt{140}}\prn{5\cdot1+5\cdot1,(-1)\cdot1+(-1)\cdot1,3\cdot1+3\cdot1} \\ &= \frac1{\sqrt{140}}\prn{5+5,(-1)+(-1),3+3} \\ &= \frac1{\sqrt{140}}(10,-2,6) }$$
or in matrix notation:
$$\align{ \mtrxof{S_1f_1,g} &\equiv \mtrxofB{\frac1{\sqrt{\lambda_1}}Tf_1,g} \\ &= \frac1{\sqrt{\lambda_1}}\mtrxof{Tf_1,g} \\ &= \frac1{\sqrt{\lambda_1}}\mtrxof{T,e,g}\mtrxof{f_1,e} \\ &= \frac1{\sqrt{70}}\pmtrx{5&5\\-1&-1\\3&3}\pmtrx{\frac1{\sqrt2}\\\frac1{\sqrt2}} \\ &= \frac1{\sqrt{70}}\frac1{\sqrt2}\pmtrx{5&5\\-1&-1\\3&3}\pmtrx{1\\1} \\ &= \frac1{\sqrt{140}}\pmtrx{5\cdot1+5\cdot1\\(-1)\cdot1+(-1)\cdot1\\3\cdot1+3\cdot1} \\ &= \frac1{\sqrt{140}}\pmtrx{5+5\\(-1)+(-1)\\3+3} \\ &= \frac1{\sqrt{140}}\pmtrx{10\\-2\\6} }$$
And define $S_2\in\linmapb{\rangspb{\sqrttat}^\perp}{\rangsp{T}^\perp}$ by
$$ S_2f_2\equiv\frac1{\sqrt{104}}(2,10,0) $$
Then
$$\align{ \mtrxof{S,f,g} &= \pmtrx{\mtrxof{S,f,g}_{:,1}&\mtrxof{S,f,g}_{:,2}} \\ &= \pmtrx{\mtrxof{Sf_1,g}&\mtrxof{Sf_2,g}} \\ &= \pmtrx{\frac{10}{\sqrt{140}}&\frac{2}{\sqrt{104}}\\-\frac{2}{\sqrt{140}}&\frac{10}{\sqrt{104}}\\\frac{6}{\sqrt{140}}&0} }$$
and
$$\align{ \mtrxof{S,f,g}\mtrxofb{\sqrttat,f}\adjt{\mtrxof{I,f,e}} &= \pmtrx{\frac{10}{\sqrt{140}}&\frac{2}{\sqrt{104}}\\-\frac{2}{\sqrt{140}}&\frac{10}{\sqrt{104}}\\\frac{6}{\sqrt{140}}&0}\pmtrx{\sqrt{70}&0\\0&0}\adjt{\pmtrx{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}}} \\\\ &= \pmtrx{\frac{10\sqrt{70}}{\sqrt{140}}&0\\-\frac{2\sqrt{70}}{\sqrt{140}}&0\\\frac{6\sqrt{70}}{\sqrt{140}}&0}\pmtrx{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}} \\\\ &= \pmtrx{\frac{10}{\sqrt{2}}&0\\-\frac{2}{\sqrt{2}}&0\\\frac{6}{\sqrt{2}}&0}\pmtrx{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}} \\\\ &= \pmtrx{\frac{10}{2}&\frac{10}{2}\\-\frac22&-\frac22\\\frac{6}{2}&\frac{6}{2}} \\\\ &= \pmtrx{5&5\\-1&-1\\3&3} \\\\ &= \mtrxof{T,e,g} }$$