Current time:0:00Total duration:19:12

0 energy points

# Unique rowspace solution to Ax = b

Video transcript

Let's say I've got an
m-by-n matrix A. That's my matrix right there. And I could just write it as a
series of n column vectors, so it could be a1, a2,
all the way to an. Now, let's say that I have
some other vector b. Let's say b is a member of
the column space of A. Remember, the column space is
just the set of all of the vectors that can be represented
as a linear combination of the columns of A,
so that means that b can be represented as a linear
combination of the columns of A. So I'll just write the constant
factors as x1 times a1 plus x2 times a2, all the way
to plus xn times an, where x1, x2, xn, they're all just
arbitrary real numbers. Another way to state this is
that that means that a, which I could write as a1, a2, all
the way to an, times some vector x1, x2, all the way
to xn, is equal to b. These two statements
are equivalent. We know that b is a member
of the column space. That means that b can be
represented as a linear combination of the columns of
A, and that this statement right here can be rewritten
this way. So you can write that the
equation Ax equals b has at least one solution x that
is a member of Rn. And the entries of x would
represent the weights on the column vectors of A to get your
linear combination b. This is all a bit of review. Now, let's draw Rn. Any solution to this equation
right here is going to be a member of Rn. Remember, this is an
m-by-n matrix. We had n columns and this has
to be a member of Rn right there, so let's draw Rn. So Rn maybe looks like
that, so that is Rn. And let's look at some of the
subspaces that we have in Rn. We have the null space. That's going to be in Rn. The null space is all of the
solutions to the equation Ax is equal to 0. That's going to be in Rn. It's all of the x's that satisfy
that equation, so let me draw that right here. So let's say I have the null
space right there, so that is the null space of A. And then what else
do we have in Rn? Well, we have the orthogonal
complement of the null space of A. Let me draw that. So we have the orthogonal
complement-- let me doing it in a different color. We have the orthogonal
complement of the null space of A, which we can also
call-- we learned this in the last video. This is also going to be equal
to the row space of A, which is also just the column space. The row space of A is the column
space of A transpose. So we have two spaces here. That is the row space of A. So I have two subsets of Rn. I have the null space and then
I have the null space's complement, orthogonal
complement, which is the row space of A. Now, we've seen in several
videos now, and I proved it I think two videos ago, that
any vector in Rn can be represented as a sum of a member
of our null space, let's call that vector n, and
let's say some vector in our row space, let's call
that vector r. Any vector in Rn can be
represented as a sum of some vector in our null space and
some vector in our row space. So any solution to this equation
is a member of Rn so it must be able to be
represented by some member of our null space and some member
of our row space. So let's write that down. So let's say x is a solution
to Ax equals b, which also means that x is a member of Rn,
so because it's a member of Rn, we can represent it as
a combination of one vector here and one vector there. So let's say that x is equal
to some vector r0, plus n0, where r0 is a member of our row
space and n0 is a member of the row space's orthogonal
complement. They are the orthogonal
complements of each other, so n0 is a member of
our null space. Fair enough. Now, one thing we might wonder
is, clearly this vector isn't a solution to Ax equals b. This vector is a solution
to Ax is equal to 0. But we might be curious as to
whether this solution right here, this member of our row
space is a solution to Ax is equal to b. This is kind of what we're
focused on in this. So let's solve for
r0 right here. So if we solve for r0, if we
subtract n0 from both sides, we get r0 is equal
to x minus n0. All I did was subtract n0 from
both sides and I switched things around. I solved for r0. Now, if we multiply, A times
r0 is equal to A times this whole thing-- let me switch
colors-- that's equal to A times x0 minus n0, which is
equal to Ax minus An0. And what is this equal to? Well, A times x, we already said
that x is a solution to Ax equals b, so this
right here is going to be equal to b. And n0 is a member of our null
space, which means it satisfies this solution right
here, that A times any member of our null space is going to
be equal to the zero vector. So that's going to be equal
to the zero vector. So you have the vector b minus
the zero vector, and you're just going to have
the vector b. So we just found out that A
times this member of our row space-- let's call that
r0, that's that guy right there maybe. A times r0 is equal to b. So this is a solution. So r0 is a solution to
Ax is equal to b. So far, it 's kind of an
interesting result that we have already. If you give me any vector here
b that is a member of our column space, then there is
going to be some member of our row space right here that is a
solution to Ax is equal to b. Now, the next question you might
wonder is, is this the only guy in our row space
that is a solution to Ax is equal to b? And to prove that,
let's assume that there's another guy here. Let's say that r1 is a member
of our row space and a solution to Ax is equal to b. Now, the row space is a valid
subspace, so if I take the sum or the difference of any two
vectors in the row space, I'll get another member
of the row space. That's one of the requirements
for being a valid subspace. So let's see this. So if I take two members of our
subspace, so if I take r1 minus r0 and I take their
difference, which is just the sum-- well, you multiply 1 times
the negative and that has to be a member of the
subspace when you're summing them, so this has to be a
member of our subspace. So this must also be a member
of our of our row space. That's because our row space
is a valid subspace. You get two of its members, you
take its difference, that also has to be a member. Fair enough. Now, let's see what
happens when you multiply this guy by A. So if I take A times r1 minus
r0, what do I get? I get A times r1 minus
A times r0. We already figured out, or for
r1 we assumed that it is a solution to Ax is equal to b,
and r0, we already found out, it is a solution to
Ax equal to b. So either of these, when
you multiply them by A, it equals b. So this equals b and that equals
b, so you get b minus b, which is the zero vector. Now, this is interesting. This tells us that r minus r0 is
a solution to the equation Ax is equal to 0, right? When I put r1 minus r0 in the
place of x right there and I multiplied it times
the A, I got 0. I got 0, which implies that r1
minus r0, that this vector is a member of our null space. So I have a vector here that's
a member of my row space, and we got that from the fact that
both of these are members of our row space and the row space
is closed under addition and subtraction, and the vector
r1 minus r0 is a member of my null space. And we've seen this several
times already. If I have a vector that is in
a subspace and it's also in the orthogonal complement of the
subspace, the null space is also the orthogonal
complement of the row space, then the only possible
vector that that can be is the zero vector. That's the only vector that's
inside of a subspace and it's orthogonal complement or a
subspace and it's orthogonal complement. These two guys are
the orthogonal complements of each other. We drew it up here. So we get that r1 one
minus r0 must be equal to the zero vector. That's the only vector that's
in a subspace and its orthogonal complement,
which implies that r1 must be equal to r0. When we take the difference,
we get the zero vector. So we have a couple of
neat results here. What do we know so far? We know that if we have some
vector b that is a member of our column space of A,
then there exists a unique member, right? We just proved the uniqueness. There exists a unique member
of the row space of A. Let me write it. Let me do it in a
different color. Of the row space of A, so this
is the row space of A such that a unique member of the
row space of A, let me call that r0. Let me do it in a
different color. I want to make this really
stand out in your brain. So we know that r0 is a member
of the row space of A such that r0 is a solution
to Ax is equal to b. It's a little bit of a complex
statement here, but it's interesting. You give me any b that's a
member of the column space of A, then there will exist a
unique member of the row space of A, that's my unique member of
the row space of A, that is a solution to Ax
is equal to b. Now, we can go further
with this. We can go further. We wrote up here that any
solution to this equation Ax is equal to b can be written as
a sum of r0 plus n0, where r0 is a member of our row space
and n0 is a member of our null space, and that's
because we have a subspace and its orthogonal complement. So any member of Rn can be
represented as a sum of a subspace and a member of the
subspace's orthogonal complement. Let me rewrite that down here. So we already said that any
solution x to Ax is equal to b can be written as a
combination-- let me write it this way-- as a combination
of r0 plus n0. Fair enough. Now, what happens if I wanted
to take the square of the length of x on both
sides of that. Let me write this down, and
you'll see why I'm writing this, because I have
another interesting result to show you. So if I were take the square
of any solution to this equation right here, well,
that's going to be the same thing as x dot x, which is the
same thing as this thing dot itself, same thing as r0
plus n0 dot r0 plus n0. And what is this equal to? This is equal to r0 dot r0 plus
n0 dot r0 plus n0 dot r0 again plus n0 dot n0. I just kind of foiled it out and
we can do that because we know the dot product exhibits
the distributive property. So this thing right here
is equal to the length of r0 squared. Now, we're going to have--
what is n0 dot r0? We don't even have to simplify
this much more. n0 is a member of our null space. r0 is a member of
our row space. Each of them is in a subspace
that is the orthogonal complement of the other, which
means that everything here dotted with anything in
here is equal to 0. So r0 dot n0 is going
to be equal to 0. These guys are orthogonal to
each other, so that's going to be equal to 0, that's going to
be equal to 0, and then you get plus-- what's this? n0 dot
n0 is just the length of the vector n0 squared. These are all vectors. And so we get the length of the
vector x squared is equal to the length of our member of
our row space squared, our unique member of our row space
squared, plus that member of our null space squared. Now, this is definitely going
to be a positive number. It's at minimum 0, but it has to
be something larger than 0, so we can say that this
quantity right here is definitely greater than or
equal to just r0 squared. Or another way to think about
it is, you give me any solution to the equation Ax is
equal to b, and the square of its length is going to be
greater than or equal to the square of r0's length. Or since both of the lengths are
always positive, you can take kind of the positive square
root and you know you won't have to switch signs
there, that the length of any solution to Ax equals b is going
to be greater than or equal to the length of r0. So that makes r0 kind of
a special solution. So now let's write our entire
statement, everything that we've learned in this video. So if b is a member of the
column space of A, then there exists a unique r0 that is a
member of the row space of A, such that r0 is a solution
to Ax is equal to b. And not only is it a solution,
it's a special solution. r0 is the solution with the
least, or no solution has a smaller length than r0. Let me write it that way. Maybe some other solution could
be equal but could have the same length. And no other solution can
have a smaller length. Maybe we could write that if you
give me any vector b that is a member of the column space
of A, then there exists a unique member of the row space
that is essentially the smallest solution. You can write small as having
the least length to Ax is equal to b, which is a
pretty neat outcome. In the next video, we'll
explore this a little bit more visually.