Orthogonal complements
Unique rowspace solution to Ax=b Showing that, for any b that is in the column space of A, there is a unique member of the row space that is the "smallest" solution to Ax=b
⇐ Use this menu to view and help create subtitles for this video in many different languages.
You'll probably want to hide YouTube's captions if using these subtitles.
- Let's say I've got an m-by-n matrix A.
- That's my matrix right there.
- And I could just write it as a series of n column vectors, so
- it could be a1, a2, all the way to an.
- Now, let's say that I have some other vector b.
- Let's say b is a member of the column space of A.
- Remember, the column space is just the set of all of the
- vectors that can be represented as a linear
- combination of the columns of A, so that means that b can be
- represented as a linear combination of
- the columns of A.
- So I'll just write the constant factors as x1 times
- a1 plus x2 times a2, all the way to plus xn times an, where
- x1, x2, xn, they're all just arbitrary real numbers.
- Another way to state this is that that means that a, which
- I could write as a1, a2, all the way to an, times some
- vector x1, x2, all the way to xn, is equal to b.
- These two statements are equivalent.
- We know that b is a member of the column space.
- That means that b can be represented as a linear
- combination of the columns of A, and that this statement
- right here can be rewritten this way.
- So you can write that the equation Ax equals b has at
- least one solution x that is a member of Rn.
- And the entries of x would represent the weights on the
- column vectors of A to get your linear combination b.
- This is all a bit of review.
- Now, let's draw Rn.
- Any solution to this equation right here is going to be a
- member of Rn.
- Remember, this is an m-by-n matrix.
- We had n columns and this has to be a member of Rn right
- there, so let's draw Rn.
- So Rn maybe looks like that, so that is Rn.
- And let's look at some of the subspaces that we have in Rn.
- We have the null space.
- That's going to be in Rn.
- The null space is all of the solutions to the equation Ax
- is equal to 0.
- That's going to be in Rn.
- It's all of the x's that satisfy that equation, so let
- me draw that right here.
- So let's say I have the null space right there, so that is
- the null space of A.
- And then what else do we have in Rn?
- Well, we have the orthogonal complement of the
- null space of A.
- Let me draw that.
- So we have the orthogonal complement-- let me doing it
- in a different color.
- We have the orthogonal complement of the null space
- of A, which we can also call-- we learned
- this in the last video.
- This is also going to be equal to the row space of A, which
- is also just the column space.
- The row space of A is the column space of A transpose.
- So we have two spaces here.
- That is the row space of A.
- So I have two subsets of Rn.
- I have the null space and then I have the null space's
- complement, orthogonal complement, which is the row
- space of A.
- Now, we've seen in several videos now, and I proved it I
- think two videos ago, that any vector in Rn can be
- represented as a sum of a member of our null space,
- let's call that vector n, and let's say some vector in our
- row space, let's call that vector r.
- Any vector in Rn can be represented as a sum of some
- vector in our null space and some vector in our row space.
- So any solution to this equation is a member of Rn so
- it must be able to be represented by some member of
- our null space and some member of our row space.
- So let's write that down.
- So let's say x is a solution to Ax equals b, which also
- means that x is a member of Rn, so because it's a member
- of Rn, we can represent it as a combination of one vector
- here and one vector there.
- So let's say that x is equal to some vector r0, plus n0,
- where r0 is a member of our row space and n0 is a member
- of the row space's orthogonal complement.
- They are the orthogonal complements of each other, so
- n0 is a member of our null space.
- Fair enough.
- Now, one thing we might wonder is, clearly this vector isn't
- a solution to Ax equals b.
- This vector is a solution to Ax is equal to 0.
- But we might be curious as to whether this solution right
- here, this member of our row space is a solution to Ax is
- equal to b.
- This is kind of what we're focused on in this.
- So let's solve for r0 right here.
- So if we solve for r0, if we subtract n0 from both sides,
- we get r0 is equal to x minus n0.
- All I did was subtract n0 from both sides and I switched
- things around.
- I solved for r0.
- Now, if we multiply, A times r0 is equal to A times this
- whole thing-- let me switch colors-- that's equal to A
- times x0 minus n0, which is equal to Ax minus An0.
- And what is this equal to?
- Well, A times x, we already said that x is a solution to
- Ax equals b, so this right here is going
- to be equal to b.
- And n0 is a member of our null space, which means it
- satisfies this solution right here, that A times any member
- of our null space is going to be equal to the zero vector.
- So that's going to be equal to the zero vector.
- So you have the vector b minus the zero vector, and you're
- just going to have the vector b.
- So we just found out that A times this member of our row
- space-- let's call that r0, that's that
- guy right there maybe.
- A times r0 is equal to b.
- So this is a solution.
- So r0 is a solution to Ax is equal to b.
- So far, it 's kind of an interesting result that we
- have already.
- If you give me any vector here b that is a member of our
- column space, then there is going to be some member of our
- row space right here that is a solution to Ax is equal to b.
- Now, the next question you might wonder is, is this the
- only guy in our row space that is a solution to
- Ax is equal to b?
- And to prove that, let's assume that
- there's another guy here.
- Let's say that r1 is a member of our row space and a
- solution to Ax is equal to b.
- Now, the row space is a valid subspace, so if I take the sum
- or the difference of any two vectors in the row space, I'll
- get another member of the row space.
- That's one of the requirements for being a valid subspace.
- So let's see this.
- So if I take two members of our subspace, so if I take r1
- minus r0 and I take their difference, which is just the
- sum-- well, you multiply 1 times the negative and that
- has to be a member of the subspace when you're summing
- them, so this has to be a member of our subspace.
- So this must also be a member of our of our row space.
- That's because our row space is a valid subspace.
- You get two of its members, you take its difference, that
- also has to be a member.
- Fair enough.
- Now, let's see what happens when you
- multiply this guy by A.
- So if I take A times r1 minus r0, what do I get?
- I get A times r1 minus A times r0.
- We already figured out, or for r1 we assumed that it is a
- solution to Ax is equal to b, and r0, we already found out,
- it is a solution to Ax equal to b.
- So either of these, when you multiply them by
- A, it equals b.
- So this equals b and that equals b, so you get b minus
- b, which is the zero vector.
- Now, this is interesting.
- This tells us that r minus r0 is a solution to the equation
- Ax is equal to 0, right?
- When I put r1 minus r0 in the place of x right there and I
- multiplied it times the A, I got 0.
- I got 0, which implies that r1 minus r0, that this vector is
- a member of our null space.
- So I have a vector here that's a member of my row space, and
- we got that from the fact that both of these are members of
- our row space and the row space is closed under addition
- and subtraction, and the vector r1 minus r0 is a member
- of my null space.
- And we've seen this several times already.
- If I have a vector that is in a subspace and it's also in
- the orthogonal complement of the subspace, the null space
- is also the orthogonal complement of the row space,
- then the only possible vector that that can
- be is the zero vector.
- That's the only vector that's inside of a subspace and it's
- orthogonal complement or a subspace and it's orthogonal
- complement.
- These two guys are the orthogonal
- complements of each other.
- We drew it up here.
- So we get that r1 one minus r0 must be
- equal to the zero vector.
- That's the only vector that's in a subspace and its
- orthogonal complement, which implies that r1
- must be equal to r0.
- When we take the difference, we get the zero vector.
- So we have a couple of neat results here.
- What do we know so far?
- We know that if we have some vector b that is a member of
- our column space of A, then there exists a
- unique member, right?
- We just proved the uniqueness.
- There exists a unique member of the row space of A.
- Let me write it.
- Let me do it in a different color.
- Of the row space of A, so this is the row space of A such
- that a unique member of the row space of A, let
- me call that r0.
- Let me do it in a different color.
- I want to make this really stand out in your brain.
- So we know that r0 is a member of the row space of A such
- that r0 is a solution to Ax is equal to b.
- It's a little bit of a complex statement here, but it's
- interesting.
- You give me any b that's a member of the column space of
- A, then there will exist a unique member of the row space
- of A, that's my unique member of the row space of A, that is
- a solution to Ax is equal to b.
- Now, we can go further with this.
- We can go further.
- We wrote up here that any solution to this equation Ax
- is equal to b can be written as a sum of r0 plus n0, where
- r0 is a member of our row space and n0 is a member of
- our null space, and that's because we have a subspace and
- its orthogonal complement.
- So any member of Rn can be represented as a sum of a
- subspace and a member of the subspace's orthogonal
- complement.
- Let me rewrite that down here.
- So we already said that any solution x to Ax is equal to b
- can be written as a combination-- let me write it
- this way-- as a combination of r0 plus n0.
- Fair enough.
- Now, what happens if I wanted to take the square of the
- length of x on both sides of that.
- Let me write this down, and you'll see why I'm writing
- this, because I have another interesting
- result to show you.
- So if I were take the square of any solution to this
- equation right here, well, that's going to be the same
- thing as x dot x, which is the same thing as this thing dot
- itself, same thing as r0 plus n0 dot r0 plus n0.
- And what is this equal to?
- This is equal to r0 dot r0 plus n0 dot r0 plus n0 dot r0
- again plus n0 dot n0.
- I just kind of foiled it out and we can do that because we
- know the dot product exhibits the distributive property.
- So this thing right here is equal to the
- length of r0 squared.
- Now, we're going to have-- what is n0 dot r0?
- We don't even have to simplify this much more. n0 is a member
- of our null space.
- r0 is a member of our row space.
- Each of them is in a subspace that is the orthogonal
- complement of the other, which means that everything here
- dotted with anything in here is equal to 0.
- So r0 dot n0 is going to be equal to 0.
- These guys are orthogonal to each other, so that's going to
- be equal to 0, that's going to be equal to 0, and then you
- get plus-- what's this? n0 dot n0 is just the length of the
- vector n0 squared.
- These are all vectors.
- And so we get the length of the vector x squared is equal
- to the length of our member of our row space squared, our
- unique member of our row space squared, plus that member of
- our null space squared.
- Now, this is definitely going to be a positive number.
- It's at minimum 0, but it has to be something larger than 0,
- so we can say that this quantity right here is
- definitely greater than or equal to just r0 squared.
- Or another way to think about it is, you give me any
- solution to the equation Ax is equal to b, and the square of
- its length is going to be greater than or equal to the
- square of r0's length.
- Or since both of the lengths are always positive, you can
- take kind of the positive square root and you know you
- won't have to switch signs there, that the length of any
- solution to Ax equals b is going to be greater than or
- equal to the length of r0.
- So that makes r0 kind of a special solution.
- So now let's write our entire statement, everything that
- we've learned in this video.
- So if b is a member of the column space of A, then there
- exists a unique r0 that is a member of the row space of A,
- such that r0 is a solution to Ax is equal to b.
- And not only is it a solution, it's a special solution.
- r0 is the solution with the least, or no solution has a
- smaller length than r0.
- Let me write it that way.
- Maybe some other solution could be equal but could have
- the same length.
- And no other solution can have a smaller length.
- Maybe we could write that if you give me any vector b that
- is a member of the column space of A, then there exists
- a unique member of the row space that is essentially the
- smallest solution.
- You can write small as having the least length to Ax is
- equal to b, which is a pretty neat outcome.
- In the next video, we'll explore this a
- little bit more visually.
Be specific, and indicate a time in the video:
At 5:31, how is the moon large enough to block the sun? Isn't the sun way larger?
|
Have something that's not a question about this content? |
This discussion area is not meant for answering homework questions.
Discuss the site
For general discussions about Khan Academy, visit our Reddit discussion page.
Flag inappropriate posts
Here are posts to avoid making. If you do encounter them, flag them for attention from our Guardians.
abuse
- disrespectful or offensive
- an advertisement
not helpful
- low quality
- not about the video topic
- soliciting votes or seeking badges
- a homework question
- a duplicate answer
- repeatedly making the same post
wrong category
- a tip or feedback in Questions
- a question in Tips & Feedback
- an answer that should be its own question
about the site
Share a tip
Suggest a fix
Have something that's not a tip or feedback about this content?
This discussion area is not meant for answering homework questions.