- Matrix vector products
- Introduction to the null space of a matrix
- Null space 2: Calculating the null space of a matrix
- Null space 3: Relation to linear independence
- Column space of a matrix
- Null space and column space basis
- Visualizing a column space as a plane in R3
- Proof: Any subspace basis has same number of elements
- Dimension of the null space or nullity
- Dimension of the column space or rank
- Showing relation between basis cols and pivot cols
- Showing that the candidate basis does span C(A)
Calculating the null space of a matrix. Created by Sal Khan.
Want to join the conversation?
- Does calculating the null space of vectors have something to do with the identity of orthogonality and perpendicularity ? Seeing as how when you dot vector a with vector b you get zero (a.b) = 0, if they are orthogonal to one another, is there a similarity ? I can't define and capture the overall concept (if there is one) which explains the relationship between the two. Also as a corollary, how does this relatesto linear combinations - in particular linear independence and dependence ?(18 votes)
- Great question,
The nullspace is very closely linked with orthogonality. Recall that the definition of the nullspace of a matrix A is the set of vectors x such that Ax=0 i.e. is the zero vector. How do we compute Ax? When we multiply a matrix by a vector we take the dot product of the first row of A with x, then the dot product of the second row with x and so on. The only way for Ax=0 is if every row of A is orthogonal to x.
From this idea we define something called the row space. It is the subspace generated by the row vectors of A. The vector x lives in the same dimension as the row vectors of A so we can ask if x is orthogonal to the row vectors. In fact, given any subspace we can always find the orthogonal complement, which is the subspace containing all the orthogonal vectors. The orthogonal complement of the row space is the null space.
Linear independence comes in when we start thinking about dimension. The dimension of a subspace generated by the row space will be equal to the number of row vectors that are linearly independent. When the row space gets larger the null space gets smaller since there are less orthogonal vectors. If an nxn matrix A has n linearly independent row vectors the null space will be empty since the row space is all of R^n.
There is much more to say but this should get you started thinking about it. Keep it up!(32 votes)
- Hey Sal. Can we consider the null space of an matrix the same as the kernel (ker) of an function?(17 votes)
- yes we can. nullspace and kernel of a matrix are the same.
both are the set of all x that satisfy Ax=0.(18 votes)
- So really isn't this just finding the set of vectors that are perpendicular to our system of equations (i.e. matrix A)?(11 votes)
- That's right. As Sal says at9:48, we're finding the set of vectors x that satisfy the equation Ax = 0. This means if you do the dot product of any of these vectors x with any of the rows of A, you'll get zero, and a zero dot product implies they're perpendicular (orthogonal).(14 votes)
- What does "augmented" mean?(3 votes)
- Augmented means to increase in size. In the linear algebra setting, it means to increase a matrix in size by adding entries on the right hand side. What the matrix is augmented by depends on what you're doing. If you're solving a system of linear equations, the vector containing the solutions is added on the RHS. If you're finding the inverse of a matrix, the matrix to be inverted is augmented by the identity matrix.(10 votes)
- Do the two vectors (1 -2 1 0) & (2 -3 0 1) have a special name??(5 votes)
- Not as such, but it's nice to tie it back to the terms we already know. Those vectors form a basis of the null space of the matrix A.(6 votes)
- Would this work if you put the matrix in echelon form and not row canonical form?(5 votes)
- With "Echelon form" meaning that you left a leading coefficient as something other than
1, then it would work, but it would make the math slightly more complicated on the last step. Then you'd have something like
7x1 = 7x3 + 14x4instead of
x1 = x3 + 2x4at the end. You'd have to divide through by the constant (in that case
7), in order to solve for x1 again. It ultimately changes nothing, but sort of adds a step.(4 votes)
- At the end he says that N(A)==N(rref(A)). Shouldn't it be N(A)==Span(N(rref(A)))?(1 vote)
- In the problem, A and rref(A) are not the same matrix. A is the matrix:
|1 1 1 1|
|1 2 3 4|
|4 3 2 1|
Whereas the matrix rref(A) is:
|1 0 -1 -2|
|0 1 2 3|
|0 0 0 0|
The point of saying that N(A) = N(rref(A)) is to highlight that these two different matrices in fact have the same null space. This means that instead of going through the process of creating the augmented matrix and carrying around all those zeros, you can find rref(A) first and then find the null space of that.(7 votes)
- Is there any difference at all between Null Space and Kernel, or is it two terms that mean exactly the same thing?(2 votes)
- They both mean almost exactly the same thing. The only difference is the usage. Kernel is used for any linear transformation while nullspace is only used with matrices. So if you have a matrix A, you can find N(A), but not ker(A) since a martix by itself is just an expression and not a linear transformation. Likewise, if you have
T(x⃑) = A x⃑, then you can take ker(T), but not N(T). You can take N(A) and ker(T), and they'll give you the same answer.
N(A) = ker(T)(7 votes)
- At7:45, why does Sal only solve for x1 and x2? Is it because they are the ones with the pivot entries?(3 votes)
- Exactly right. Since they are pivot vectors it's easy to get them in terms of the non pivot entries. So you could pick any number for x4 and any number for x3, then if you perform vector addition with that it would give you the corresponding x1 and x2, let me know if you don't see how to do that. Now if you multiply the matrix A by the 4x2 matrix formed by the nullspace you will get a 0 matrix of dimensions 3x2
You can also get the same result if you multiply any row from matrix A by a constant, so the first row <1, 1, 1, 1> would become <2, 2, 2, 2> if you multiplied it by 2, or if you multiplied any COLUMN in the nullspace by a constant. So the column <1, -2, 1, 0> could become <3, -6, 3, 0> if you multiplied the column by 3. Doing any of this will still result in a 0 matrix.
Let me know if that doesn't make sense or something.(1 vote)
- Is there any material in this course (linear algebra) that requires the use of the term "kernel"?(2 votes)
- It depends on what textbook you use. In a book called "Algebra" by Michael Artin, it defines kernel and nullspace rather differently. Let us not worry about this for khanacademy.
On khanacademy, and at the introductory linear algebra level, I would highly recommend a sort unofficial convention (which is also suggested by math stack exchange):
If the context is only matrices, use the term "Null space" when talking about the set of vectors x such that Ax=0.
If talking about abstract linear transformations T: V -> W, avoid using the term "Null space", and instead use "Kernel" when describing the set of vectors x such that T(x)=0.(3 votes)
In the last video, I spoke somewhat theoretically about what a null space is and we showed that it is a valid subspace. But in this video let's actually calculate the null space for a matrix. In this case, we'll calculate the null space of matrix A. So null space is literally just the set of all the vectors that, when I multiply A times any of those vectors, so let me say that the vector x1, x2, x3, x4 is a member of our null space. So when I multiply this matrix times this vector I should get the 0 vector. I should get the vector. And just to make a few points here, this has exactly 4 columns. This is a 3 by 4 matrix, so I've only legitimately defined multiplication of this times a four-component vector or a member of Rn. Let me call this X. And this is our vector X. This is a member of R4. It has four components. And then when you multiply these, we need to produce a 0 vector. The null space is the set of all the vectors, and when I multiply it times A, I produce the 0 vector. And what am I going to get? I'm going to have one row times this and that's going to be the first entry, then this row times, that's the second entry, and then the third row. So I should have three 0's So my 0 vector is going to be the 0 vector in R3. So how do we figure out the set of all of these x's that satisfy this? Let me just write our formal notation. The null space of A is the set of all vectors that are a member of -- we generally say Rn, but this is a 3 by 4 matrix, so these are all the vectors that are going to be members of R4, because I'm using this particular A, such that my matrix A times any of these vectors is equal to the 0 vector. In this case it's going to be 0 vector in R3. So how do we do this? Well, this is just a straight up linear equation. We can write it that way. If we were to actually perform the matrix multiplication, we get 1 times x1. Let me write it here. Let me do it in a different color. 1 times x1, plus 1 times x2, plus 1 times x3, plus 1 times x4 is equal to this 0 there. So that times that is equal to that 0. And then this times this should be equal to that 0. So 1 times x1, so you get x1, plus 2 times x2, plus 3 times x3, plus 4 times x4 is going to be equal to that 0. And then finally we have that times this vector should be equal to that 0. So the dot product of this row vector with this column vector should be equal to that 0. So you get 4x1. 4x1 plus 3x2 plus 2x1 plus 2x3 plus x4 is equal to 0. 4x1 plus 3x2 plus 2x3 plus x4 is equal to 0. You just have to find the solution set to this and we'll essentially have figured out our null space. Now, we've figured out the solution set to systems of equations like this. We have three equations with four unknowns. We can do that. We can represent this by an augmented matrix and then put that in reduced row echelon form. Let's do that. I can represent this problem as the augmented matrix. 1, 1, 4. 1, 2, 3. 1, 3, 2. and then 1, 4, 1. And then I augment that with the 0 vector. And the immediate thing you should notice is we took the pain of multiplying this times this to equal that, and we wrote this as a system of equations, but now we want to solve the system of equations, we're going back to the augmented matrix world. What does this augmented matrix look like? Well, this is just our matrix A right there. That's just matrix A right there, that's just the 0 vector right there. And to solve this, and we've done this before, we're just going to put this augmented matrix into row echelon form. What you're going to find is when you put it into row echelon form, this right side's not going to change at all, because no matter what you multiply or subtract by, you're just doing it all times 0, so you just keep ending up with 0. So as we put this into reduced row echelon form, were actually just putting matrix A into reduced echelon form. So let me do that, instead of just talking about it. So let me start off by keeping row 1 the same. Row 1 is 1, 1, 1, 1, 0. And then I want to eliminate this 1 right here, so let me replace row 2 with row 2 minus row 1. So 1 minus 1 is 0. 2 minus 1 is 1. 3 minus 1 is 2. 4 minus 1 is 3. 0 minus 0 is 0. You can see the 0's aren't going to change. And then let me replace this guy with 4 times this guy, minus this guy. So I can only get rid of this. So 4 times 1 minus 4 is 0. 4 times 1 minus 3 is 1. 4 times 1 minus 2 is 2. 4 times 1 minus 1 is 3. 4 times 0 minus 0 is 0. Now I want to get rid of, if I want to put this in reduced row echelon form, I want to get rid of that term and that term. So let me keep my middle row the same. My middle row is 0, 1, 2, 3. So that's 0 on the augmented side of it, although these 0's are never going to change, it's really just a little bit of an exercise just to keep writing them. And my first row, let me replace it with the first row minus the second row, so I can get rid of this 1. So 1 minus 0 is 1. 1 minus 1 is 0. 1 minus 2 is minus 1. 1 minus 3 is minus 2. And 0 minus 0 is 0. And let me replace this last row with the last row minus the middle row. So 0 minus 0 is 0. 1 minus 1 is 0. 2 minus 2 is 0. I think you see where this is going. 3 minus 3 is 0. And obviously 0 minus 0 is 0. So this system of equations has been reduced, just by doing reduce row echelon form, this problem. If I just rewrite this right here, this can be written as a system equations of x1 minus x3 minus x4, right? The 0 x2's is equal to 0. And then this second row right here, there's no x1, you just have an x2, plus 2x3, plus 3x2 is equal to 0, and this obviously gives me no information whatsoever. And so I can solve this. I can solve this for x1 and x2, and what do I get? I get x1 is equal to x3 plus x4. Actually, I made a mistake here. This is x1 minus x3 minus 2 times x4 is equal to 0. So if I rewrite this, I get x1 is equal to x3 plus 2x4. And then I get x2. Let me do that in green. x2 is equal to minus 2 x3 minus 3x2. So if I wanted to write the solution set to this equation, if I wanted to write it in terms of this, I could write x1, x2, x3, x4 is equal to -- what's x1 equal to? It's equal to x3 times 1 plus x4 times 2. Right? I just got this right here from this equation right here. x1 is equal to 1 times x3, plus 2 times x4. That's just that, right there. Now, x2 is equal to x3 times minus 2, plus x4 times minus 3. What am I doing? I'm losing track of things. This x2 right here is x2 plus 2x3 plus 3x4 is equal to 0. So x2 is equal to minus 2x3, minus 3x4. Right. Like that. Sorry, my brain isn't completely in the problem, I'm making these silly mistakes. But I think you understand this now. So then what is x3 equal to? Well, it's just equal to 1 times x3, plus 0 times x4, right? x3 is equal to x3. And what's x4 equal to? It's equal to 0 times x3 plus 1 times x4. So all of the vectors in R4, these are a member R4, which satisfy the equation, our original equation, ax is equal to 0, can be represented as a linear combination of these two vectors, of those two vectors, right? These are just random scalars that are a member of-- We can pick any real number for x3 and we could pick any real number for x4. So our solution set is just a linear combination of those two vectors. What's another way of saying a linear combination of two vectors? Let me write this. The null space of A, which is just a solution set of this equation, it's just all the x's that satisfy this equation, it equals all of the linear combinations of this vector and that vector. What do we call all the linear combinations of two vectors? It's the span of those two vectors. So it equals the span of that vector and that vector. Of the vector 1, minus 2, 1, 0, and the vector 2, minus 3, 0, 1. And this is our null space. Before letting you go, let me just point out one interesting thing right here. We represented our system of equation like this and we put it into reduced row echelon form, so this is A and this is 0. This right here is, let me make sure I have some space, let me put it right here. That right there is the reduced row echelon form of A. And so where essentially this equation, this is a linear equation that is trying to solve this problem. The reduced row echelon form of A times our vector x is equal to 0. So, all the solutions to this are also the solutions to our original problem, to our original ax is equal to 0. So what's the solution to this? All the x's that satisfy this, these are the null space of the reduced row echelon form of A. Right? So here are all of the x's, this is the null space, this problem, if we find all of the x's here, this is the null space of the reduced row echelon form of our matrix A. But we're saying that this problem is the same problem as this one, right? So we can write that the null space of A is equal to the null space of the reduced row echelon form of A. And that might seem a little bit confusing, hey, why are you even writing this out, but it's the actually very useful when you're trying to calculate null spaces. So we didn't even have to write a big augmented matrix here. We can say, take our matrix A, put it in reduced row echelon form and then figure out it's null space. We would have gone straight to this point right here. This is the reduced row echelon form of A, and then I could have immediately solved these equations, right? I would have just taken the dot product of the reduced row echelon form or, not the dot product, the matrix vector product of the reduced row echelon form of A with this vector, and I would've gotten these equations, and then these equations would immediately, I can just rewrite them in this form, and I would have gotten our result. But anyway, hopefully you found that reasonably useful.