Orthogonal Matrices
( w 1 → T w 2 → T w 3 → T ) ( v 1 → v 2 → v 3 → ) = ( w 1 → T v 1 → w 1 → T v 2 → w 1 → T v 3 → w 2 → T v 1 → w 2 → T v 2 → w 2 → T v 3 → w 3 → T v 1 → w 3 → T v 2 → w 3 → T v 3 → ) Suppose that { v ^ 1 , v ^ 2 , . . . , v ^ n } is an orthonormal basis A = ( v ^ 1 . . . v ^ n ) Then, A T = A − 1 ( A T A ) i j = v ^ i ⋅ v ^ j = { 1 , i = j 0 , i ≠ j \begin{pmatrix}
\overrightarrow{w_1}^T \\
\overrightarrow{w_2}^T \\
\overrightarrow{w_3}^T \\
\end{pmatrix}
\begin{pmatrix}
\overrightarrow{v_1} & \overrightarrow{v_2} & \overrightarrow{v_3}
\end{pmatrix} =
\begin{pmatrix}
\overrightarrow{w_1}^T \overrightarrow{v_1} & \overrightarrow{w_1}^T \overrightarrow{v_2} & \overrightarrow{w_1}^T \overrightarrow{v_3} \\
\overrightarrow{w_2}^T \overrightarrow{v_1} & \overrightarrow{w_2}^T \overrightarrow{v_2} & \overrightarrow{w_2}^T \overrightarrow{v_3} \\
\overrightarrow{w_3}^T \overrightarrow{v_1} & \overrightarrow{w_3}^T \overrightarrow{v_2} & \overrightarrow{w_3}^T \overrightarrow{v_3} \\
\end{pmatrix} \\
\text{Suppose that } \{\hat{v}_1, \hat{v}_2, ..., \hat{v}_n\} \text{ is an orthonormal basis} \\
A = \begin{pmatrix}
\hat{v}_1 & ... & \hat{v}_n
\end{pmatrix} \\
\text{Then, } A^T = A^{-1} \\
(A^T A)_{ij} = \hat{v}_i \cdot \hat{v}_j = \begin{cases}
1, i = j \\
0, i \neq j
\end{cases} ⎝ ⎛ w 1 T w 2 T w 3 T ⎠ ⎞ ( v 1 v 2 v 3 ) = ⎝ ⎛ w 1 T v 1 w 2 T v 1 w 3 T v 1 w 1 T v 2 w 2 T v 2 w 3 T v 2 w 1 T v 3 w 2 T v 3 w 3 T v 3 ⎠ ⎞ Suppose that { v ^ 1 , v ^ 2 , ... , v ^ n } is an orthonormal basis A = ( v ^ 1 ... v ^ n ) Then, A T = A − 1 ( A T A ) ij = v ^ i ⋅ v ^ j = { 1 , i = j 0 , i = j Lines of Best Fit (LSRL)
A x → = b → ( 1 0 1 1 1 2 1 3 ) ( β 0 β 1 ) = ( β 0 β 1 ( 0 ) β 0 β 1 ( 1 ) β 0 β 1 ( 2 ) β 0 β 1 ( 3 ) ) = ( 2 4 1 3 ) A T A = ( 1 1 1 1 0 1 2 3 ) ( 1 0 1 1 1 2 1 3 ) = ( 4 6 6 14 ) A T b → = ( 1 1 1 1 0 1 2 3 ) ( 2 4 1 3 ) = ( 10 15 ) ( 4 6 10 6 14 15 ) → ( 1 0 2.5 0 1 0 ) ( β 0 β 1 ) = ( 2.5 0 ) which corresponds to the line y = 2.5 A \overrightarrow{x} = \overrightarrow{b}
\begin{pmatrix}
1 & 0 \\
1 & 1 \\
1 & 2 \\
1 & 3
\end{pmatrix}
\begin{pmatrix}
\beta_0 \\
\beta_1
\end{pmatrix} =
\begin{pmatrix}
\beta_0 & \beta_1(0) \\
\beta_0 & \beta_1(1) \\
\beta_0 & \beta_1(2) \\
\beta_0 & \beta_1(3)
\end{pmatrix} =
\begin{pmatrix}
2 \\
4 \\
1 \\
3
\end{pmatrix} \\
A^T A =
\begin{pmatrix}
1 & 1 & 1 & 1 \\
0 & 1 & 2 & 3
\end{pmatrix}
\begin{pmatrix}
1 & 0 \\
1 & 1 \\
1 & 2 \\
1 & 3
\end{pmatrix} =
\begin{pmatrix}
4 & 6 \\
6 & 14 \\
\end{pmatrix} \\
A^T \overrightarrow{b} =
\begin{pmatrix}
1 & 1 & 1 & 1 \\
0 & 1 & 2 & 3
\end{pmatrix}
\begin{pmatrix}
2 \\
4 \\
1 \\
3
\end{pmatrix} =
\begin{pmatrix}
10 \\
15 \\
\end{pmatrix} \\
\left(
\begin{array}{cc|c}
4 & 6 & 10 \\
6 & 14 & 15\\
\end{array}
\right) \rightarrow
\left(
\begin{array}{cc|c}
1 & 0 & 2.5 \\
0 & 1 & 0 \\
\end{array}
\right)
\begin{pmatrix} \beta_0 \\ \beta_1 \end{pmatrix} = \begin{pmatrix} 2.5 \\ 0 \end{pmatrix} \text{ which corresponds to the line } y = 2.5 A x = b ⎝ ⎛ 1 1 1 1 0 1 2 3 ⎠ ⎞ ( β 0 β 1 ) = ⎝ ⎛ β 0 β 0 β 0 β 0 β 1 ( 0 ) β 1 ( 1 ) β 1 ( 2 ) β 1 ( 3 ) ⎠ ⎞ = ⎝ ⎛ 2 4 1 3 ⎠ ⎞ A T A = ( 1 0 1 1 1 2 1 3 ) ⎝ ⎛ 1 1 1 1 0 1 2 3 ⎠ ⎞ = ( 4 6 6 14 ) A T b = ( 1 0 1 1 1 2 1 3 ) ⎝ ⎛ 2 4 1 3 ⎠ ⎞ = ( 10 15 ) ( 4 6 6 14 10 15 ) → ( 1 0 0 1 2.5 0 ) ( β 0 β 1 ) = ( 2.5 0 ) which corresponds to the line y = 2.5