Inverses

We saw earlier that products of matrices and vectors have a definition that can be understood through the lens of transformation. But what about division by a matrix? The only intuitive definition we can think of might be dividing every coefficient in an $m×n$ matrix, but that would run counter to our understanding of multiplication, which is defined on matrices of size $m×n$ and $n×m$, where each component becomes a dot product of its corresponding row and column.

Instead, we might want to think about division in terms of multiplication. Remember that multiplying any number by a fraction with numerator 1 is a division by the denominator of that fraction. And also recall that any fraction with numerator 1 is the inverse of multiplication by the denominator, in the sense that if we multiply 1 by the denominator, multiplying by the fraction will undo the multiplication and yield 1 again.

$1×a×\frac{1}{a}=1$

For the sake of notational convenience, we usually refer to such fractions as negative exponents, since we can exploit the fact that multiplying two numbers together adds their powers and the sum of 1 and -1 is zero.

$1×{a}^{1}×{a}^{-1}={a}^{0}=1$

Note that not every number has an inverse. In particular, if what we are multiplying by is zero then there is no way that you can use a function to get the number back, since zero times anything is zero. Expressed as fractional notation, it would not make much sense either:

$1×0×\frac{1}{0}=?$

In those cases, we say that the number has no inverse, since it would be the same thing as dividing by zero.

We can express the same thing for matrices as well - the product of any matrix and its inverse (if the inverse exists) will always be the indentity matrix, $I$

${A}^{-1}×A=I$

So if we can find ${A}^{-1}$ then by premultiplying a matrix by it, we are effectively dividing by $A$.

Again, note that not every matrix has an inverse. Recall that some transformations squash all of space on to a line or a plane:

In this case, it would be impossible to find a matrix that would go back to our original space, since multiple points within the space all could have ended up in the same position on the plane. We also said that at least for this 3x3 transformation that it had an overall volume of zero and thus the determinant was zero. We will see shortly why a zero determinant means that an inverse cannot exist formally.

Now for computing inverses. We will start with the 2x2 case. In this instance, I am just going to throw down a formula, but be aware that the formula is not a general or even recursive definition for how an inverse is calculated, but is really just a shortcut for the 2x2 case:

$\frac{1}{detA}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right)$

You would be forgiven for thinking that formula had been pulled out of nowhere, so lets break it down into its components by focusing on an example: a matrix that scales by 2 in both directions and rotates counterclockwise.

$\left[\begin{array}{cc}0& 2\\ -2& 0\end{array}\right]$

Now, consider just multiplying the transformed basis vectors by the rearranged matrix we saw earlier. Does this get us back to the standard basis vectors?

$\left[\begin{array}{cc}0& -2\\ 2& 0\end{array}\right]$

Unfortunately that did not quite do what we wanted. We rotated back to our basis vectors, but now they are four times as big! The problem with just rearranging the matrix like this is that it did not undo the change in the area, in fact, the determinant of the transformation was preserved so by multiplying by the new matrix, we just made space twice more expanded.

This is the reason why in order to find the inverse, we also need to undo the change in the determinant by scaling by the inverse of the determinant.

$\frac{1}{0×0-2×-2}×\left(\begin{array}{cc}0& -2\\ 2& 0\end{array}\right)$$\frac{1}{4}×\left(\begin{array}{cc}0& -2\\ 2& 0\end{array}\right)$

Now, if we do that, the transformation takes us back to our basis vectors. Note that the fact that we multiply by the inverse of the determinant means that some matrices are by definition non-invertable, because we cannot multiply by the inverse of zero.

Now if this section followed the others, it would give a generalized technique for extending this knowledge into multiple dimensions. Unfortunately, there is no general mechanism for computing inverses of$n×m$ matrices. Instead, we will explore a more algorithmic approach using Elementary Row Operations.

Recall that by using Elementary Row Operations that we were able to take a complex looking transformation and reduce it down to the identity matrix. If we were able to represent those operations as a matrix, that matrix would be the inverse of the original matrix.

Consider the same 3x3 transformation we row-reduced above:

$\left[\begin{array}{ccc}1& -1& 0\\ -1& -1& 0\\ 2& 1& 2\end{array}\right]$

We will row-reduce it again, but this time, we will apply the same operations to the identity matrix and observe how that transformation affects an already transformed unit cube. So, side by side:

$\left[\begin{array}{ccc}1& -1& 0\\ -1& -1& 0\\ 2& 1& 2\end{array}\right]$$\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$

First, subtract 2 times the first row from the third.

$\left[\begin{array}{ccc}1& -1& 0\\ -1& -1& 0\\ 0& 3& 2\end{array}\right]$$\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ -2& 0& 1\end{array}\right]$

Now subtract the second row from the first.

$\left[\begin{array}{ccc}2& 0& 0\\ -1& -1& 0\\ 0& 3& 2\end{array}\right]$$\left[\begin{array}{ccc}1& -1& 0\\ 0& 1& 0\\ -2& 0& 1\end{array}\right]$

Then add half of the first row to the second.

$\left[\begin{array}{ccc}2& 0& 0\\ 0& -1& 0\\ 0& 3& 2\end{array}\right]$$\left[\begin{array}{ccc}1& -1& 0\\ 0.5& 0.5& 0\\ -2& 0& 1\end{array}\right]$

Add three times the second row to the third

$\left[\begin{array}{ccc}2& 0& 0\\ 0& -1& 0\\ 0& 0& 2\end{array}\right]$$\left[\begin{array}{ccc}1& -1& 0\\ 0.5& 0.5& 0\\ 0.5& 1.5& 1\end{array}\right]$

Clean up the matrix by multipying the first row by $\frac{1}{2}$, the second row by -1 and the third row by $\frac{1}{2}$.

$\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$$\left[\begin{array}{ccc}0.5& -0.5& 0\\ -0.5& -0.5& 0\\ -0.25& 0.75& 0.5\end{array}\right]$

And with that, we have found the inverse