Main content

## Linear dependence and independence

Current time:0:00Total duration:17:38

# More on linear independence

## Video transcript

I think by now we have a
reasonable sense of what linear dependence means. So let's just do a slightly
more formal definition of linear dependence. So we're going to say that a set
of vectors-- Let me just define my set of vectors. Let me call my set s
of vectors, v1, v2, all the way to vn. I'm going to say that they
are linearly dependent. If and only if. So sometimes it's written if, if
with a lot of f's in there. So sometimes it's written
if and only if. Sometimes it's shown like an
arrow in two directions. If and only if I can satisfy
this equation, I can find a set of constants c1 times v1. I can take a linear combination
of my vectors all the way to cn vn, that satisfy
the equation that I can create this into the 0 vector. Sometimes it's just written as
a bold 0, and sometimes you could just write it -- I
mean we don't know the dimensionality of this vector. It would be a bunch of 0's. We don't know how many actual
elements are in each of these vectors, but you get the idea. My set of vectors is linearly
dependent-- remember I'm saying dependent, not
independent --is linearly dependant, if and only if I can
satisfy this equation for some ci's where not all of
them are equal to 0. This is key, not all are 0. Or you could say it
the other way. You could say at least
one is non-zero. So how does this gel with what
we were talking about in the previous video where I said
look, a set is linearly dependent if one of the of
vectors can be represented by the combination of the
other vectors? Let me write that down. In the last few I said, look,
one vector can be -- Let me write it this way. One vector being represented
by the some of the other vectors, I can just write
it like this. I can write it a little
bit more math-y. In the last video, I said that
linear dependence means that-- let me just pick an arbitrary
vector, v1. Let's say that v1, you know
this is arbitrary, v1 one could be represented
by some combination of the other vectors. Let me call them a1 times v --
let me be careful -- a2 times v2 plus a3 times v3 plus all
the way up to an times vn. This is what we said in
the previous video. If this is linear dependence,
any one of these guys can be represented as some combination
of the other ones. So how does this imply that? In order show this if and only
if, I have to show that this implies that and I have to show
that that implies this. So this is almost a trivially
easy proof. Because if I subtract v1 from
both sides of this equation I get 0 is equal to minus 1 v1
plus a2 v2 plus a3 v3 all the way to an vn. And clearly I've just
said, well, this is linearly dependent. That means that I can represent
this vector as a sum of the other vectors, which
means that minus 1 times v1 plus some combination of the
other vectors is equal to 0, which means that I've been able
to satisfy this equation, and at least one of my constants
is non-zero. So I've shown you that, if I
can represent one of the vectors by a sum of the other
ones, then this condition is definitely going to be true. Now let me go the other way. Let me show you if I have this
situation that I can definitely represent one
of the vectors as the sum of the others. So let's say that
this is true. And one of these constants,
remember it's not just this, it's at least one,
is non-zero. So let me just assume, just for
the sake of simplicity-- I mean these are all arbitrary. I'll do it in a new color. Let me do it in the magenta. Let me assume that c1
is not equal to 0. If c1 is not equal to 0, then
I can divide both sides of this equation by c1. And what do I get? I get v1 plus c2 over c1 v2 plus
all the way up to cn over c1 is equal to 0. And then I can multiply both
sides of this, or I could add negative v1 to both sides of
this equation or subtract v1 from both sides. And I get c2 over c1 v2 plus all
the way up to cn over c1 vn-- there's a vn here --is
equal to minus v1. Now if I just multiply both
sides of this by negative 1, I get a minus, and all these
become minuses and this becomes a plus. And I just showed you that
if at least one of these constants is non-zero, that I
can represent my vector v1 as some combination of
the other vectors. So we're able to go
this way too. If this condition is true, then
I can represent one of the vectors as a combination
of the others. If I can represent one of those
vectors as a combination of the others, then this
condition is true. Hopefully that kind of proves
that these two definitions are equivalent. Maybe it's a little
bit of overkill. Let's apply that definition
now, to actually test. You might say, hey Sal, why'd you go
through all of this effort? I went through all of this
effort because this is actually a really useful way
to test whether things are linearly independent
or dependent. Let's try it out. Let's use our newly
found tool. Let's say I have the
set of vectors-- Let me do it up here. I want to be efficient
with my space usage. So let's say I have the set
of vectors 2,1 and 3,2. And my question to you is, are
these linearly independent or are they linearly dependent? In order for them to be linearly
dependent, that means that if some constant times 2,1
plus some other constant times this second vector,
3,2 where this should be equal to 0. Where these both aren't
necessarily 0. Before I go up for this problem,
let's remember what we're going to find out. If either of these are non-zero,
if c1 or c2 are non-zero, then this implies
that we are dealing with a dependent, linearly
dependent set. If c1 and c2 are both 0, if the
only way to satisfy this equation -- I mean you can
always satisfy it by sitting everything equal to 0. But if the only way to satisfy
it is by making both of these guys 0, then we're
dealing with a linearly independent set. Let's try to do some math. And this'll just take us back
to our Algebra 1 days. In order for this to be true,
that means 2 times c1 plus 3 times c2 is equal to -- when I
say this is equal to 0, it's really the 0 vector. I can rewrite this as 0,0. So 2 times c1 plus 3
times c2 would be equal to that 0 there. And then we'd have 1 times
c1 plus 2 times c2 is equal to that 0. And now this is just a system,
two equations, two unknowns. A couple of things
we could do. Let's just multiply this
top equation by 1/2. If you multiply it by 1/2 you
get c1 plus 3/2 plus 3/2 c2 is equal to 0. And then if we subtract the
green equation from the red equation this becomes 0. 2 minus 1 and 1/2-- 3/2 is 1 and
1/2 --of this is just 1/2 c2 is equal to 0. And this is easy to solve. c2 is equal to 0. So what's c1? Well, just substitute this back
in. c2 is equal to 0. So this is equal to 0. So c1 plus 0 is equal to 0. So c1 is also equal to 0. We could have substituted
it back into that top equation as well. So the only solution to this
equation involves both c1 and c2 being equal to 0. So they both have to be 0. So this is a linearly
independent set of vectors. Which means that neither
of them are redundant of the other one. You can't represent one as a
combination of the other. And since we have two vectors
here, and they're linearly independent, we can actually
know that this will span r2. The span of my r vectors
is equal to r2. If one of these vectors was
just some multiple of the other, than the span would have
been some line within r2, not all of. But now I can represent any
vector in r2 as some combination of those. Let's do another example. Let me scroll to the right,
because sometimes this thing, when I go too far down, I
haven't figured out why, when I go too far down it
starts messing up. So my next example is
the set of vectors. So I have the vector 2,1. I have the vector 3,2. And I have the vector 1,2. And I want to know are these
linearly dependent or linearly independent. So I go to through
the same drill. I use that little theorem that
I proved at the beginning of this video. In order for them to be linearly
dependent there must be some set of weights that
I can multiply these guys. So c1 times this vector plus c2
times this vector plus c3 times that vector, that will
equal the 0 vector. And if one of these is non-zero
then we're dealing with a linearly dependent
set of vectors. And if all of them are 0,
then it's independent. Let's just do our
linear algebra. So this means that 2 times c1
plus 3 times c2 plus c3 is equal to that 0 up there. And then if we do the bottom
rows-- Remember when you multiply a scalar times a vector
you multiply it by each of these terms. So c1 times 1. 1c1 plus 2c2 plus 2c3
is equal to 0. There's a couple of giveaways
on this problem. If you have three
two-dimensional vectors, one of them is going to
be redundant. Because, in the very best case,
even if you assume that that vector and that vector are
linearly independent, then these would span r2. Which means that any point,
any vector, in your two-dimensional space can
be represented by some combination of those two. In which case, this is going to
be one of them because this is just a vector in
two-dimensional space. So it would be linearly
dependent. And then, if you say, well,
these aren't linearly independent then, they're just
multiples of each other. In which case, this would
definitely be a linearly dependent set. When you see three vectors that
are each only vectors in r2, that are each
two-dimensional vectors, it's a complete giveaway that this
is linearly dependent. But I'm going to show it to you
using our dependent, using our little theorem here. So I'm going to show you that I
can get non-zero c3's, c2's, and c1's such that I
can get a 0 here. If all of these had to be 0-- I
mean you can always set them equal to 0. But if they had to be equal to
0, then it would be linearly independent. Let me just show you. I can just pick some
random c3. Let me pick c3 to be equal
to negative 1. So what would these two
equations reduce to? I mean you have just three
unknowns and two equations, it means you don't have enough
constraints on your system. So if I just set c3-- I just
pick that out of a hat. I could have picked
c3 to be anything. But if I set c3 to be equal to
negative 1, what do these equations become? You get 2c1 plus 3c2 minus
1 is equal to 0. And you get c1 plus 2c2
minus 2 is equal to 0. Right? 2 times minus 1. What can I do here? If I multiply this second
equation by 2, what do I get? I get 2 plus 4c2 minus
4 is equal to 0. And now let's subtract this
equation from that equation. So the c1's cancel out. 3c2 minus 4c2 is minus c2. And then minus 1 minus minus 4,
so that's minus 1 plus 4. That's plus 3 is equal to 0. And so we get our -- Let me make
sure I got that right. We have a minus 1
minus a minus 4. So plus 4. So we have a plus 3. So that is a minus 2. So minus c2 is equal to minus
3 or c2 is equal to 3. And if c2 is equal to 3 and c3
is equal to minus 1-- Let's just substitute here, so we
get c1 plus 2 times c2, so plus 6, plus 2 times c3. So minus 2 is equal to 0. c1 plus 4 is equal to 0. c1 is equal to minus 4. I'm giving you a combination
of c's that will give us a 0 vector. If I multiply minus 4 times our
first vector, 2,1, that's c1, plus 3 times our second
vector, 3,2 minus 1 times our third vector, 1,2 this
should be equal to 0. Let's verify it just for fun. Minus 4 times 2 is minus
8 plus 9 minus 1. Yeah, that's minus 9 plus 9. That's 0. Minus 4 times plus 6 minus
2 that's also 0. So we've just shown a linear
combination of these vectors, where actually none of
the constants are 0. But all we had to show was
that at least one of the constants had to be non-zero,
and we actually showed all three of them were. But at least one of these
had to be non-zero. And I was able to satisfy this
equation, I was able to make them into the zero vector. So this shows, this proves,
that this is a linearly dependent set of vectors. Which means one of the
vectors is redundant. And you can never just say,
oh, this is the redundant vector, because I can represent
this as combination of those two. You could just as easily pick
this guy as the redundant vector, and say, hey, I can
represent this guy as the sum of those two. There's not one bad apple
in the bunch. Any of them can be represented
by the combination of some other, by all of the
rest of them. So hopefully you have a better
intuition of linear dependence and independence. Maybe I'll continue. I'll do a few more examples
in the next video.