# Linear Independence

Much ado gets made about linear independence, probably because it makes up quite a few questions where the answer is not immediately obvious. It is also a bit of terminology that gets in the way of understanding. Its hard to imagine vectors being dependent on each other, surely they can be manipulated independently?

When we talk about linear independence what we are actually talking about is whether a vector in a set of vectors actually gives us the freedom to move in another dimension or whether it is trapped in the same dimension already described by other vectors.

Of course, even that is a little misleading. You could have vectors with non-zero numbers in every dimension but they still may be linearly dependent. So what we are actually talking about is whether that vector in combination with other vectors gives us access to more space. This is best illustrated in the two dimensional case with a visual explanation.

$\left[\begin{array}{cc}1& 2\\ 2& 4\end{array}\right]$

These two vectors are not linearly independent. The reason why is that the first vector describes the line $y=2x$ and the second vector describes the line $2y=4x$.

But that is the exact same line! If we were to add any scalar multiple of these two vectors, you would still get another vector that describes the line $y=2x$. So we say that they are Linearly Dependent

Linear dependence is kind of obvious in the 2-vectors, 2-dimensions case, because it basically only comes up when the set of vectors are just scalar multiples of each other. But what happens if you have 3 vectors in a two dimensional space?

$\left[\begin{array}{c}1\\ 2\end{array}\right]$$\left[\begin{array}{c}1\\ 1\end{array}\right]$$\left[\begin{array}{c}4\\ 5\end{array}\right]$

These vectors are still linearly dependent, even though none of them line on the same line. The reason is that now instead of lying on the same line, they all line on the same plane, which by definition, is the entire 2D co-ordinate space. By scaling the lines describes by $y=x$ and$2y=x$, adding them together and so on. I can reach any point in 2D space already. I do not need $5y=4x$ in order to get access to any more space.

For example, lets make the vector $\left(4,5\right)$ using only $\left(1,1\right)$ and $\left(1,2\right)$.

$1×\left(\left(1,2\right)-\left(1,1\right)\right)+4×\left(1,1\right)=1×\left(0,1\right)+4×\left(1,1\right)=\left(4,5\right)$

This is a little more subtle when we move into three dimensions. Just because you have three vectors, it doesnt mean that they are all linearly independent.

Obviously, you can get a case where they represent the same line

$\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right]$$\left[\begin{array}{c}2\\ 4\\ 6\end{array}\right]$$\left[\begin{array}{c}-1\\ -2\\ -3\end{array}\right]$

But you might also get a case where they, represent the same plane

$\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right]$$\left[\begin{array}{c}1\\ 0\\ 1\end{array}\right]$$\left[\begin{array}{c}2\\ 1\\ 2\end{array}\right]$

It is not immediately obvious, but adding the third vector to the other two does not get you outside of the plane that the other two are describing. So they are linearly dependent.

So we can get rid of the third vector and still be describing the same plane.

$\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right]$$\left[\begin{array}{c}1\\ 0\\ 1\end{array}\right]$