Main content
Course: Linear algebra > Unit 1
Lesson 7: Null space and column space- Matrix vector products
- Introduction to the null space of a matrix
- Null space 2: Calculating the null space of a matrix
- Null space 3: Relation to linear independence
- Column space of a matrix
- Null space and column space basis
- Visualizing a column space as a plane in R3
- Proof: Any subspace basis has same number of elements
- Dimension of the null space or nullity
- Dimension of the column space or rank
- Showing relation between basis cols and pivot cols
- Showing that the candidate basis does span C(A)
© 2024 Khan AcademyTerms of usePrivacy PolicyCookie Notice
Visualizing a column space as a plane in R3
Determining the planar equation for a column space in R3. Created by Sal Khan.
Want to join the conversation?
- Sorry, I don't have a particular spot in the video to refer to, but here goes:
I have been watching the Linear Algebra sessions from the beginning and thought I had a good grip on what was going on then, when I got here and we had been working with columns for a few lectures, the question came to mind:
How does what we are doing in column spaces relate to the origin of the original matrix. I mean the rows of the matrix represent a linear equation, but the column of a matrix only references the same variable in each of the linear equations.
Hoping for an explanation
Mark(14 votes)- If you have Ax = b, then b must be a linear combination of the columns of A. Say it is 1 times the first column, plus 2 times the second column... then x is the vector (1,2). For, say, a 2x2 matrix A there is a row way of looking at it, and the lines represented by each row intersect at x, plus a column way of looking at it where each column is a vector and the numbers in x tell you how much to multiply each column by so that their sum results in the b vector. It is 2 ways to do the same problem or think about it.(11 votes)
- I don't get it. Is it always a plane? The column space is only a plane if there are 2 basis vectors. What if there are 3?(7 votes)
- Yes I had the same thought. My guess: if there are three basis vectors, the column space is a volume in R3. If there is one, the column space is a line in R3. As there ar two in this example, it is a plane in R3. Would like confirmation of this, though?(11 votes)
- For the second approach, what about the constraints in the top 2 rows?
e.g., that x_2 - x_3 - x_4 = 2x - y.
Is it not possible that this would alter the plane constraint?
We already heuristically know that the solution should be a plane. Is this why these cases were not (or did not have to be) considered?(10 votes)- I was wondering the same thing. The last row of the rref of Ax=u (x and u both variable vectors), which is one of the results of solving Ax=u, says unambiguously that 5x - y - z = 0; so do we need more than that?(1 vote)
- So, since spanning subspaces must always contain the zero vector, then all spanning subspaces intersect the origin?(5 votes)
- All valid subspaces contain the zero vector and therefore pass through the origin.(1 vote)
- @7:10sal ,cross products the two positions vectors and those vectors are not "ON" the plane they are just touching the plane . so how can the cross product be a normal to the plane ?
for those having the same question
at 13.34 , Sal clearly explains why he was able to get the normal vector by doing a cross product. ie the two vectors are also part of the plane. now why are they part of the plane because all the points of the vectors can be obtained from their linear combination. eg the equation av1 + bv2 where v1,v2 are vectors and a,b are scalars a can be zeroed out to get the end point of vector v2 and then b can be incremented from zero to get all the points in vector v2. the same thing can be done for v1(6 votes)- since, origin is lying in the plane so the position vector is also lying in the plane(2 votes)
- Since the rref of the A only has two identity vectors, e1 and e2, doesn't the mean that the column space spans R^2 and not R^3? meaning the plane would be R^2? This kinda goes for the whole video(2 votes)
- R^2 is a description used for the set of all vectors with 2 components, and R^3 is the set of all vectors with 3 components. As these vectors have 3 components they are members of the R^3 set. The column space might then be visualised as a 2d plane inside this set, but it is not R^2 as the vectors still have the extra component.(6 votes)
- At11:11you said the plane has to intersect (0,0,0) because Subspace always contain the zero vector. But i thinks it a little mistake right? Because you said vector doesn't contain position information? Is the real reason is you draw the vectors at the standard point? I also think any other parallel plane with the one that you found can contain the same set of vector.(4 votes)
- I have similar questions. I suspect, but don't expect, the answer will become clear later.(1 vote)
- At13:00, it is said that the normal vector is the result of the cross product of the two basis vectors. Just to confirm: Is possible then to generalize this by saying that the cross product of any two vectors that are a "basis" (therefore lie on the plane they describe) generate a normal vector? Thus, this wouldn't work with "position vectors" (which do not lie on the plane), right? Finally, is this the main difference between "position vectors" and "basis vectors"?(2 votes)
- Basis vectors are simply position vectors that lie in the subspace, in this case the plane, which are the minimum vectors required to define that subspace. The cross product will find the vector that is orthogonal to that subspace, but you aren't necessarily limited to the cross product of the basis vectors since the other vectors also lie in the subspace. They're simply superfluous vectors in that subspace that can be defined using linear combinations of the basis vectors. You will still end up with the same plane.
Here's proof if you don't believe me (v1 x v3): http://i.imgur.com/S5IvArH.png
Of course, if you just take any old position vector that points to a point on the plane it's not going to work since it doesn't lie along the subspace plane.(3 votes)
- At5:27, you create an invalid 3-dimensional coordinate space! When constructing axes, z should always be in the direction of x cross y! Wouldn't be as important if we were just in High School algebra, but in Linear algebra it's kind of important!(3 votes)
- Can we generalize that basis of a vector only contains column vectors that fall into pivot variables? In other words, in the future, can we skip the process of proving x3 and x4 is the combination of x1 and x2, and just say the basis of this vector is a x1 and x2 column space since they are the only pivot variables?(2 votes)
- Notice that in the 4x3 matrix from the last video that a n y set of 2 of the column vectors of the matrix A is a basis for the column space (since all the pairs are l. i.). So I think you're partly right.(2 votes)
Video transcript
In the last video, I started
with this matrix right here, and right from the get go, we
said the span of this matrix is just the span of the column vectors
of it, and I just wrote it right there. But we weren't clear whether
this was linearly independent, and if it's not linearly independent,
it won't be a sufficient basis. And then we go off and
we figure out the null space of A. We find out that the null space
of A contains more than just the zero vector. It's just the span of these two
vectors here, which means that these columns are not
linearly independent. We saw that several
videos ago. And we used that information
that they're not linearly independent to try to make them
linearly independent by getting rid of the
redundant ones. So we were able to get rid of
this guy and this guy, because these were essentially the
columns associated with the free variables. And we were able to do it using
this little technique down here, where we set one of
these equal to 0, the other one equal to negative
1 and then solved for the pivot variables. And then we set the other one
equal to 0 and the other one equal to negative 1 and solved
for the pivot variables. And you could imagine
that this is a generalizable process. If you have a bunch of free
variables, you can set all of them but one to equal 0, and
then that one that you didn't set to 0, you set it to
equal negative 1. And you can express it as a sum
of the pivot variables, where the pivot variables are a function of the free variables. In general, this would be
a quick way of doing it. We had to do it more
slowly over here. If I have some matrix A, and I
want to figure out the basis for the column space, the column
space is just the span of these, but if I wanted a
linearly independent basis, I need to figure out some set
of these that are linearly independent. What I can do is put this guy
into reduced row echelon form. When I put them in reduced row
echelon form, and I did that over here, this is the reduced
row echelon form of A, I can look at the variables
that are associated with the pivot entries. So this is x1. Let me scroll up a little bit. This is associated
with x1, right? When you multiply this times x1,
you get this column times x1, this column times x2, this
column times x3, this column times x4 like that. When you look at just regular A,
when you look at just your matrix A, it's the same thing. If you were to write Ax equal
to 0, this column would be associated with x1, this column
would be associated with x2, x3 and x4 like that. What you can do is you put it
in reduced row echelon form. You say which columns have
my pivot entries or are associated with pivot
variables? You say, OK, x1 and x2 are
associated with pivot variables, or they are the pivot
variables, and they're associated with these first
two columns, and so those first two columns would be a
basis for the column space. How did I get this? Am I just making up
things on the fly? Well, no! It all comes from the reality
that you can always construct a situation where the vectors
associated with the free variables you can write them as
linear combinations of the vectors associated with the
pivot variables, and we did a special case of that
last time. But a very quick and dirty way
of doing it, and I don't know if it's actually dirty, is just
take your matrix, put it in it reduced row echelon form,
and you say, look, this column and this column
are associated with my free variables. Therefore, this column and this
column must be associated with my free variables. The solution sets are the same
to Ax equal to 0 or the reduced row echelon form
of Ax is equal to 0. So they're the same. So if this column and this
column are associated with free variables, so are this
column and this column, which means that they can be expressed
just by judiciously selecting your values of your
free variables as linear combinations of the columns
associated with the pivot variables, with the pivot
entries, which are that column and that column. So this column and this column
would be a basis for A, and we see that. We see that all the
way down here. 1, 2, 3 and 1, 1, 4, We did a
lot of work and we got all the way there, and we said this
is a basis for the column span of A. Now, doing all of that work,
let's see if we can actually visualize what the column
space of A looks like. I have a strange feeling I might
have said column span a couple of times, but the
column space, what does it look like? Well, there's a couple of
ways to think about what it looks like. One way is we can say, look,
this span this is 2-- that's a member of R3. That's a vector in R3 and
this is a vector in R3. Let me draw my x, z and-- well,
normally it's drawn-- this is normally y, x, and
z-axes in R3, if I'm want to represent them as
three-dimensional space. Then the vector 1, 2, 3 might
look like this where it's one, one, two, one, two, three,
so we go out one down here, then up three. So the vector will look like
that in its standard form. That's that one right there. And the vector 1, 1, 4 would be
one, one and go up four, so it might look something
like this. It's hard to actually draw
them very well in three dimensions, but you
get the idea. But the column space is the span
of these, so all of the linear combinations
of these two guys. So all of the linear
combinations of these two guys are going to create
a plane that contains these two vectors. If you just sum these guys up in
multiple combinations, you can get any vector up there. If you just add them
up, you'll get that vector right there. If you add this guy plus 2 times
that, you'll get some vector up here. So if you view them as position
vectors, they'll essentially form
a plane in R3. But let's see if we can get the
equation for that plane. Well, how can we do that? Well, we know that we can figure
out the equation of a plane based on the fact that
a normal vector dotted with any-- let me write my normal
vector like this. The normal vector dotted with
any position vector specifying a position on the plane. So let me call that x minus any
point on the plane or any vector position on the plane. So I could do that minus
the vector 1, 2, 3 has to be equal to 0. And we can use this information
to figure out the equation for this plane. But what is a normal vector? How can we find a normal
vector to this plane? So this would be a vector. Let me see if I can draw
this in a way that doesn't confuse the issue. If the plane is like that,
the normal vector would come out like that. So how can I create
a normal vector? Well, we learned that you take
the cross product of any two vectors in R3, and the cross
product I've only defined so far in R3, and I will get a
vector that's normal to both of those vectors. So let's take the
cross product. This is a nice way of thinking
about it, because it's really integrating everything that
we've covered so far. So let me define my normal
vector to be equal to 1, 2, 3 cross 1, 1, 4. And what does this equal? So my first term,
I ignore that. I get 2 times 4 minus
3 times 1. 2 times 4 is 8. 2 times 4 minus 3 times 1. 8 minus 3. Then for my second row, I have
1 times 4, and my temptation is do 1 times 4 minus
3 times 1. But you reverse it. You do 3 times 1, so it's
3, minus 1 times 4. We've done that multiple
times. You might want to review the
cross product video if that seems strange. You ignore the middle row, and
you normally do 1 times 4 minus 3 times 1, but the
middle row you switch. We're only defining it for R3,
so instead, we do 3 times 1 minus 1 times 4. And then finally for the last
row, we ignore it, and we say 1 times 1, which is 1, minus 2
times 1, which is minus 2. And this is equal to the vector
5, minus 1, minus 1, which by definition of the cross
product, and I've shown this to you multiple times,
is normal to both of these vectors, and so it'll be
normal to any linear combination of these
two vectors. So now that we have our normal
vector, we can define the traditional equation
for the plane. So we now know that our normal
vector 5, minus 1, minus 1, that I got by taking the cross
product of our basis vectors dot any vector in our plane. So let me just write
any vector. Let me just write it x, y, z. So x, y, z since that's how
I defined my axes up here. This was the x-axis. x, y and z. x, y, z minus-- I just
picked one of these. I could have picked either of
them-- minus 1, 2, 3 has got to be equal to 0. So what's this? This is going to be equal to--
let me write it a little smaller, a little neater-- 5,
minus 1, minus 1 dot-- what's this guy going to be? x minus 1, y minus 2, and
z minus 3 has got to be equal to 0. And what's the dot product? It's 5 times x minus 1 plus-- or
maybe I should say minus 1, so it's plus minus 1 times y
minus 2 plus minus 1 times z minus 3 is equal to 0. That's just the definition
of our dot product. If I simplify this, I get 5x
minus 5 minus y plus 2 minus z plus 3 is equal to 0. You have 2 plus 3 is 5 minus
5, so those all cancel out. Those will equal 0. And we get 5x minus y minus
z is equal to 0. This plane in R3 is the
column space of A. So we've now shown you that
it's truly a plane in A. And actually, it makes
sense that this plane intersects the origin. If you set x, y, and
z equal to 0, it satisfies this equation. And that makes sense, because we
said that a column space of a matrix has to be a valid
subspace, and a valid subspace has to contain the
zero vector. And in R3 that's the
coordinate 0, 0, 0. Now, what I want to do now is
see if we can get at the same answer going a completely
different way, or approaching it in a completely
different way. Let me get my original A,
which I've forgotten. I've marked it up a good
bit, but let me just copy and paste it. So that's my original
A right there. Let me copy it. Let me paste it. Nope. That's not what I
wanted to do. Let's see, so my original
A-- I copied and pasted the wrong thing. Let me do it a little-- don't
want to waste your time. Edit, copy, edit, paste. There you go and let me scroll
this down and get to a point that's relatively clean. Bring my A down. I've used a lot of space. So here you go. This is my original
A right there. And what I want to do is see
if I can get this result completely different. I got this result by figuring
out the basis of the column span, finding a normal vector by
taking the cross product of our two basis vectors, and then
using the dot product of the normal vector with the
difference-- this vector right here, where you take any vector
on our plane minus one of our basis vectors,
that's to find some vector in the plane. This is some vector
in the plane. So any vector in the plane
dotted with my normal vector is going to be equal to 0. And actually, I should probably
make a side note here, that the only reason I
was able to say that the normal vector is a cross
product of my two basis vectors, is because I knew that
these two basis vectors, not only do they specify some
point on the plane-- so let's say that this guy right here
is this blue vector. Not only does he specify some
point on the plane right there, but the vector lies
completely on the plane. How did I know that? Because I knew from the get go
that the 0, 0 vector is in my span, right? I knew that if I draw this guy
in just standard position, the point 0, 0, 0, is in my span,
and I know its end point is in the span, so this whole vector
has to be in my plane, and likewise, this whole vector
has to be in my plane. So if I take the cross product,
anything normal to these guys or any combination
of these guys is going be normal to the plane, and we got
this result right here. But let me take this right
here and use our other definition of column span. Our other definition, or it's
really an equivalent definition, is all of the valid
solutions to Ax where x is a member of Rn. Or another way we could think of
it is, we could view it as all of the valid b's where Ax
is equal to b, and x is a member of Rn. These are equivalent
statements. I'm just defining b here to be
Ax, so these are equivalent statements. But let's run with this
a little bit. So let's say that I define my
b, so b is going to be a vector in R3, right? We already have an intuition
like that. When I take Ax, I get a b
that's equal to x, y, z. And I want to figure out what x,
y's and z's can I get valid solutions for? So if I take my vector A and
then let me multiply it times-- well, actually, the
best way to do it, I think we're used to it right now. If I'm solving the equation
Ax is equal to b, I can essentially just create the
augmented matrix, where I have the matrix A and I can augment
it with b, and put this in reduced row echelon form, and
that'll essentially represent the solution set. So let me do that. So if I just augment this matrix
right here with b, so I write x, y, z. So this is A augmented with b. This is A, this is b. let me put this in reduced row
echelon form and find the solution set. And these are x, y, and z's
that define a valid b. So what do I get? The first thing I want to do,
and we've done this exercise before, let's keep my
first row the same. 1, 1, 1, 1, and I get an x. And let's replace our second row
with the second row minus the first row. Actually let me do
it this way. Let me replace the second row
with 2 times the first row minus the second row. So 2 times the first row minus
the second row, we're going to get a 2x minus y up there. And then 2 times
1 minus 2 is 0. 2 times 1 minus 1 is 1. 2 times 1 minus 4 is minus 2. 2 times 1 minus 3 is minus 1. Fair enough. And now let me replace my third
row with the third row minus 3 times the first row. So we're going to do the third
row minus-- no, let me do it this way. It's the third row minus
3 times the first row. So I'm just doing the b column
first, because I can remember what I did. The third row minus 3
times the first row. 3 minus 3 times 1 is 0. 4 minus 3 times 1 is 1. 1 minus 3 times 1 is minus 2. And then 2 minus 3 times
1 is minus 1. Now, I could go all the way to
reduced row echelon form, but something interesting is
already happening. So let me from the get go try
to zero out this third row. And the best way to zero out
this third row is to just replace the third row. So the first row-- well,
I won't even write the first row. The second row is 0, 1, minus
2, minus 1, and 2x minus y. I'm not even going to worry
about the first row right now. But let's replace the third row,
just in our attempt to go into reduced row echelon form. Let's replace it with
the second row minus the third row. So you get 2x minus
y minus z plus 3x. I just took this minus this. So minus z plus 3x. So 0 minus 0 is 0. 1 minus 1 is 0. Minus 2 minus minus 2 is
0, and that's also 0. So we're only going to have a
valid solution to Ax equals b if this guy right here
is equal to 0. What happens if he's
not equal to zero? Then we're going to have a bunch
of zeroes equaling some number, which tells us that
there's no solution. So if I pick a b where this guy
does not equal zero, then I'll have no solution. If this guy equals 5, if I pick
x, y, and z's such as that this expression is equal to
5, then Ax equal to b will have no solution, because it
will have 0 is equal to 5. So this has to equal 0. So 2x minus y minus z plus 3x
must be equal to 0 in order for b to be valid, in order for
b to be a member of the column space of A, in order for
it to be a valid vector that Ax can become, or the
product A times x can become for some x. So what does this equal to? If we add the 2x plus the 3x,
I get 5x minus y minus z is equal to 0, which is the exact
same outcome we got when we figured out the basis vectors. We said oh, you know what? The basis vectors, they have
to be in the column space themselves by definition. So let me find a normal vector
to them both by taking the cross product. I did that, and I said the cross
product times any valid vector in our space minus one of
the basis vectors has to be equal to zero, and then
I got this equation. Or we could have done
it the other way. We could've actually literally
solved this equation setting our b equal to this. We said what b's will give
us a valid solution? And our only valid solution will
be obtained when this guy has to be equal to zero, because
the rest of his row became zero. And when we set that equal
to zero, we got the exact same equation. So, hopefully, you found this
to be mildly satisfying, because we were able to tackle
the same problem from two different directions
and get the same result, and it kind of shows you
the beauty of linear algebra, how it all starts to fit together.