Linear algebra/Orthogonal matrix


 * This article contains excerpts from Wikipedia's Orthogonal matrix.
 * If complex numbers are involved, see Unitary matrix.

A real square matrix is orthogonal (orthogonal ) if and only if its columns form an orthonormal basis in a Euclidean space in which all numbers are real-valued and dot product is defined in the usual fashion. An orthonormal basis in an N dimensional space is one where, (1) all the basis vectors have unit magnitude.

Fundamental properties

 * A real square matrix is orthogonal if and only if its columns form an orthonormal basis on the Euclidean space $ℝ^{n}$, which is the case if and only if its rows form an orthonormal basis of $ℝ^{n}$.
 * The determinant of any orthogonal matrix is +1 or −1. But the converse is not true; having a determinant of ±1 is no guarantee of orthogonality. An orthogonal matrix with a determinant equal to +1 is called  a special orthogonal matrix.
 * An orthogonal matrix can always be diagonalized over the complex numbers to exhibit a full set of eigenvalues, all of which must have (complex) modulus 1.
 * All permutation matrices are orthogonal (but the converse is not true.)
 * All orthogonal matrices are unitary (but the converse is not true.)
 * Under the operation of multiplication, the $n × n$ orthogonal matrices form the orthogonal group known as $O(n)$.

Three important results that are easy to prove
Among the first things a novice should learn are those that are easy to prove.

Orthonormal basis vectors are hiding in plain sight
Theorem:


 * If the rows of a square matrix form an orthonormal set of (basis) vectors,
 * then the transpose of that matrix is its own inverse $$(\mathbf{M}^T= \mathbf{M}^{-1} )$$

Visual understanding
Suppose the rows of a matrix form an orthonormal set of basis vectors, as shown in the i-th row in matrix A to the right. The ij-the element of the product AB takes the dot product of the i-th row of A with the j-th column of matrix B, as shown in the upper part of the diagram. in the diagram's upper part, the j-th column is higlighted in yellow. In the diagram's lower part, matrix B is replaced by it's transpose, which shifts the elements in column j to a row (highlighted in cyan.) This establishes that the product of A with the transpose of B creates elements that are the dot product of rows of A with rows of B.

If A is a orthogonal matrix and B is its transpose, this procedure creates matrix elements that are dot products among the rows of the orthogonal matrix.

Rigorous proof
This proof illustrates how subscripts are used to manipulate and understand tensors.

1. Suppose


 * $$\mathbf{v_i} = \Sigma_j v_j \,\mathbf{\hat e_j} = \Sigma_j M_{ij}\,\mathbf{\hat e_j}$$ is the i-th element of a orthonormal set of basis vectors.
 * Here $$\mathbf{\hat e_j}$$ are the original unit vectors used to define the new set of unit vectors that extract from the rows of matrix $$\mathbf{\underline\underline M}$$

2. Now we relabel how we write the sums for $$\mathbf{v_i}$$ and $$\mathbf{v_j}$$ as follows:


 * $$\mathbf{v_i} = \Sigma_\alpha M_{i\alpha}\,\mathbf{\hat e_\alpha}$$


 * $$\mathbf{v_j} = \Sigma_\beta M_{j\beta}\,\mathbf{\hat e_\beta}$$


 * Hint: In the first of these two equations, I replace $$j$$ by $$\alpha$$because summed variables can be changed at will. Sometimes they are called "dummy variables" because they "do not speak" after the sum is done.  For example, summing n from 1 to 3 equals 1+2+3, which is the same as summing m from 1 to 3. In the second one I relabeled my dummy variable as $$\beta$$ because the same dummy variable cannot serve two purposes in a single expression.

3. This yields the following expression for the dot product between our two vectors:


 * $$\mathbf{v_i}\cdot\mathbf{v_j}\cdot=

\left (\sum_\alpha M_{i\alpha}\,\mathbf{\hat e_\alpha}             \right )\cdot \left ( \sum_\beta M_{j\beta}\,\mathbf{\hat e_\beta} \right )$$


 * $$\mathbf{v_i}\cdot\mathbf{v_j}\cdot=

\left (\sum_\alpha M_{i\alpha}\,\mathbf{\hat e_\alpha}\right )\cdot \left ( \sum_\beta M_{j\beta}\,\mathbf{\hat e_\beta} \right ) =\sum_{\alpha\beta}\,(\mathbf{\hat e_\alpha}\,\cdot\,\mathbf{\hat e_\beta})\,M_{i\alpha} M_{j\beta}\, $$

4. This last term introduces the Kronecker delta symbol:


 * $$\mathbf{v_i}\cdot\mathbf{v_j}\cdot=

\left (\sum_\alpha M_{i\alpha}\,\mathbf{\hat e_\alpha}\right )\cdot \left ( \sum_\beta M_{j\beta}\,\mathbf{\hat e_\beta} \right ) =\sum_{\alpha\beta} M_{i\alpha} M_{j\beta}\, \underbrace{\mathbf{\hat e_\alpha}\,\cdot\,\mathbf{\hat e_\beta}}_{\delta_{\alpha\beta}} =\sum_{\alpha} M_{i\alpha} M_{j\alpha}\, $$

The last term almost looks like the product of the matrix with itself. It can be turned into a product using the transpose on the second term in the product, using $$M_{j\alpha}=M^T_{\alpha j}.$$

5. If $$\underline\underline M$$ is orthogonal, then $$\,\underline\underline M^T=\underline\underline M^{-1},$$ and we conclude that the rows of $$\,\underline\underline M$$ (i.e., the vectors $$\mathbf{v_i})$$ form an orthonormal collection of vectors (i.e. a "rotated" basis for the vector space.)


 * $$\mathbf{v_i}\cdot\mathbf{v_j}\cdot=

\left (\sum_\alpha M_{i\alpha}\,\mathbf{\hat e_\alpha}\right )\cdot \left ( \sum_\beta M_{j\beta}\,\mathbf{\hat e_\beta} \right ) =\sum_{\alpha} M_{i\alpha} M_{j\alpha}\, =\sum_{\alpha} M_{i\alpha} M^T_{\alpha j}\, =\sum_{\alpha} M_{i\alpha} M^{-1}_{\alpha j}\, =\delta_{ij} $$

Change of basis for tensors
If a matrix is used to rotate vectors, then use it twice to rotate tensors

A common use of the orthogonal matrix is to express a vector in one reference frame into a "rotated" frame.

Here, we let $$\underline\underline M$$ denote any matrix (i.e. "tensor"), while $$\underline\underline R$$ is any orthogonal matrix (typically a rotation.) Let $$\underline v$$ and $$\underline p$$ be two vectors, and let $$\underline v'$$ and $$\underline p'$$ represent the same vectors in a rotated reference frame.


 * Theorem
 * If  $$\underline v' = \underline\underline R \cdot \underline v$$,   then:  $$ \underline\underline M' =\underline\underline R\cdot \underline\underline M \cdot \underline\underline R^{-1}$$


 * Proof
 * 1) Define $$\underline p = \underline\underline M \cdot \underline v.$$
 * 2) Assume $$\underline v' = \underline\underline R \cdot\underline v$$ and $$\underline p' = \underline\underline R \cdot\underline p.$$
 * 3) Do some tensor algebra and express $$\underline p'$$ in terms of $$\underline v'.$$

In this context, the only difference between the tensor and scalar algebras is that with tensors, vector's do not always commute: $$\underline\underline A\cdot\underline\underline B -\underline\underline B\cdot\underline\underline A$$ does not always vanish.

Derivation of the rotation tensor
The rotation matrix usually the first orthogonal matrix students encounter. While it is conceptually easier to rotate vectors than to rotate a coordinate system, it is algebraically easier to rotate a coordinate system. From the figure, the unit vectors in a rotated reference frame obey:

$$\begin{align} \hat x= & \hat x' \cos\theta -\hat y'\sin\theta \qquad &\hat y=& \hat x' \sin\theta +\hat y'\cos\theta \end{align}$$

Students will quickly see the sine and cosine components in this equation, but the minus sign might seem confusing. It comes from the fact that $$\hat x$$ has a negative component when projected along the $$\hat y'$$ direction. Now express the vector $$\underline V$$, first in the unprimed coordinate system, then in primed:

$$\underline V= V_x\hat x +\underline V_y\hat y$$

To complete the proof, substitute the expressions that expressed the $$(\hat x, \hat y)$$ unit vectors in terms of the $$(\hat x', \hat y')$$ unit vectors:

$$\underline V= V_x \overbrace{ \left(\hat x' \cos\theta - \hat y'\sin\theta \right)}^{\hat x}  +V_y \overbrace{ \left( \hat x' \sin\theta + \hat y'\cos\theta  \right)}^{\hat y} $$ $$\underline V = +V_x\hat x' \cos\theta - V_x\hat y'\sin\theta+V_y \hat x' \sin\theta +V_y\hat y'\cos\theta$$

$$\underline V = +    \underbrace{\left(V_x\cos\theta +  V_y\sin\theta \right)}_{V_x'}      \hat x' +     \underbrace{\left(- V_x\sin\theta + V_y\cos\theta \right)}_{V_x'}    \hat y' $$

This latter expression solves our problem, as we were seeking an expression of the form, $$\underline V = V_x'\hat x'+V_y'\hat y'.$$

Note how in this formalism, there is no distinction between the primed and unprimed vector $$\underline V\,.$$ This tends to confuse everyone, including the author. Such confusion can be avoided when writing a textbook or article. But in the free-wheeling world of both scientific literature, as well as wikis, such chaos cannot be avoided. That's why it is good to carefully read books.

Going back to the notation of many WMF pages, we have the following formula for the components of a vector if the coordinate system is rotated by $$\theta$$ about the z axis: $$ \begin{bmatrix}V'_x \\ V'_y\end{bmatrix}= \begin{bmatrix}\cos\theta & \sin\theta \\-\sin\theta & \cos\theta\end{bmatrix}\, \begin{bmatrix}V_x \\V_y\end{bmatrix} $$