Eigenvectors

Now that we know about Eigenvalues, we can talk about Eigenvectors and what they mean for Eigenvalues.

Like Eigenvalues, Eigenvectors translate to own vectors. Unlike Eigenvalues, it is far easier to give a geometric interpretation of what they Eigenvectors are. They are the vectors forming a basis for all vectors that do not have their direction changed when a transformation is applied to them.

For instance, when we consider the transformation $\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix}$ , which just scales everything by a factor of two, we can pretty easily say that $\begin{bmatrix} 1 & 0 \end{bmatrix}$ and $\begin{bmatrix} 0 & 1 \end{bmatrix}$ are the Eigenvectors, because the transformation as a whole does not change the direction of space. It only scales it. And of course, those two vectors are the basis for all of 2D space.

In other cases, it is not so obvious what the Eigenvectors might be. For instance, the transformation $\begin{bmatrix} 1 & 2 \\ 0 & 2 \end{bmatrix}$ has two Eigenvectors, but we would not immediately be able to say what they were:

Unsurprisingly, Eigenvectors and Eigenvalues are linked. Remember when we were computing the Eigenvalues, we were using an equation like this:

det (\begin{matrix} x_{1} - λ & x_{2} \\ y_{1} & y_{2} - λ \end{matrix}) = 0

$\det \begin{pmatrix} x_1 - \lambda & x_2 \\ y_1 & y_2 - \lambda \end{pmatrix} = 0$

If we zoom out a little and call our matrix $A$ and recall the Identity Matrix $I$ , then what we were really doing was this:

det (A - λ \cdot I)

$\det (A - \lambda \cdot I)$

Like solving for a zero-determinant, if we solve for vectors in the null space of $A - \lambda \cdot I$ , i.e, the vector $v$ in: $(A - \lambda \cdot I)v = 0$ , then we will find the Eigenvectors. Remember how we said the Eigenvectors are those vectors which do not change their direction? Here is proof of that.

(A - λ \cdot I) v = 0

$(A - \lambda \cdot I)v = 0$

A v - λ \cdot I v = 0

$Av - \lambda \cdot Iv = 0$

A v = λ \cdot I v

$Av = \lambda \cdot Iv$

From here it can be observed that $\lambda \cdot I$ is just a uniform scaling transformation. So what we are saying is that whatever $A$ does to $v$ , is the same as what $\lambda \cdot I$ does to the vector $v$ .

Solving for vector $v$ is not anything we have not seen before. In essence, we are just solving for the null space of $\begin{bmatrix} x_1 - \lambda & y_1 \\ x_2 & y_2 - \lambda \end{bmatrix}$ , for some $\lambda$ that we already worked out as an Eigenvalue before.

Suppose we have the matrix $\begin{bmatrix} 1 & 2 \\ 0 & 2 \end{bmatrix}$ , with the Eigenvalues $\lambda = 1$ and $\lambda = 2$ . That gives us two possible Eigenvectors to solve for.

First, solve for $\lambda = 1$

[\begin{matrix} 1 - 1 & 2 \\ 0 & 2 - 1 \end{matrix}]

$\begin{bmatrix} 1 - 1 & 2 \\ 0 & 2 - 1 \end{bmatrix}$

[\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

$\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ =

[\begin{matrix} 0 \\ 0 \end{matrix}]

$\begin{bmatrix} 0 \\ 0 \end{bmatrix}$

[\begin{matrix} 0 & 2 \\ 0 & 1 \end{matrix}]

$\begin{bmatrix} 0 & 2 \\ 0 & 1 \end{bmatrix}$

[\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

$\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ =

[\begin{matrix} 0 \\ 0 \end{matrix}]

$\begin{bmatrix} 0 \\ 0 \end{bmatrix}$

This matrix cannot be reduced any further by elementary row operations. Multiplying out the first row, we have $0 \cdot v_1 + 2 \cdot v_2 = 0$ . This means that $v_1$ is a free variable and we express $v_2$ in terms of it, eg:

0 \cdot v_{1} = - 2 \cdot v_{2}

$0 \cdot v_1 = -2 \cdot v_2$

which means that $v_2$ is just zero, so we can express a vector like so:

v_{1} \cdot (1, 0)

$v_1 \cdot (1, 0)$

The span of which is just $(1, 0)$ which is our first Eigenvector, for Eigenvalue $\lambda = 1$ . Lets now have a look at $\lambda = 2$

[\begin{matrix} 1 - 2 & 2 \\ 0 & 2 - 2 \end{matrix}]

$\begin{bmatrix} 1 - 2 & 2 \\ 0 & 2 - 2 \end{bmatrix}$

[\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

$\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ =

[\begin{matrix} 0 \\ 0 \end{matrix}]

$\begin{bmatrix} 0 \\ 0 \end{bmatrix}$

[\begin{matrix} - 1 & 2 \\ 0 & 0 \end{matrix}]

$\begin{bmatrix} -1 & 2 \\ 0 & 0 \end{bmatrix}$

[\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

$\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ =

[\begin{matrix} 0 \\ 0 \end{matrix}]

$\begin{bmatrix} 0 \\ 0 \end{bmatrix}$

In this case we have no free variables and an equation by examining the first row again:

- 1 \cdot v_{1} + 2 \cdot v_{2} = 0

$-1 \cdot v_1 + 2 \cdot v_2 = 0$

- 1 \cdot v_{1} = - 2 \cdot v_{2}

$-1 \cdot v_1 = -2 \cdot v_2$

v_{1} = 2 \cdot v_{2}

$v_1 = 2 \cdot v_2$

So, we can express another vector in terms of $v_2$ , that is, $v_2(2, 1)$ , which is our second Eigenvector.

Recall that for some transformations, we had less than $n$ solutions for Eigenvalues, but those solutions had higher multiplicity, $m$ . In such cases, there may be up to $m$ linearly independent vectors in the span of solutions for the Eigenvectors for that Eigenvalue. In general, where an Eigenvalue has multiplicity $m$ then you can find up to $m$ vectors, but they are not guaranteed to be linearly independent. Take for example the following matrix which shears only in the x direction:

[\begin{matrix} 1 & 1 & 0 & 1 \end{matrix}]

$\begin{bmatrix} 1 & 1 & 0 & 1 \end{bmatrix}$

The characteristic equation for this Matrix is:

det (\begin{matrix} 1 - λ & 2 \\ 0 & 1 - λ \end{matrix}) = 0

$\det \begin{pmatrix} 1 - \lambda & 2 \\ 0 & 1 - \lambda \end{pmatrix} = 0$

(1 - λ)^{2} - 2 \cdot 0 = 0

$(1 - \lambda)^2 - 2 \cdot 0 = 0$

(- λ + 1)^{2} = 0

$(-\lambda + 1)^2 = 0$

λ^{2} - 2 λ + 1 = 0

$\lambda^2 - 2\lambda + 1 = 0$

(λ - 1)^{2} = 0

$(\lambda - 1)^2 = 0$

λ = 1

$\lambda = 1$

Here we have a single Eigenvalue with algebraic multiplicity 2. If we were to solve for the Eigenvectors with this Eigenvalue, we would have:

[\begin{matrix} 1 - 1 & 1 \\ 0 & 1 - 1 \end{matrix}]

$\begin{bmatrix} 1 - 1 & 1 \\ 0 & 1 - 1 \end{bmatrix}$

[\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

$\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ =

[\begin{matrix} 0 \\ 0 \end{matrix}]

$\begin{bmatrix} 0 \\ 0 \end{bmatrix}$

[\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix}]

$\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}$

[\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

$\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ =

[\begin{matrix} 0 \\ 0 \end{matrix}]

$\begin{bmatrix} 0 \\ 0 \end{bmatrix}$

v_{2} = 0

$v_2 = 0$

And so we would have a single vector, $(0, 1)$

The dimension of the span of the corresponding linearly independent Eigenvectors for an Eigenvalue is called the geometric multiplicity of that Eigenvalue. If the sums of algebraic multiplicities and geometric multiplicities for all Eigenvalues of a matrix are equal to each other, then the matrix is said to be Diagonalizable.