Main content

## Orthonormal bases and the Gram-Schmidt process

Current time:0:00Total duration:27:04

# Example using orthogonal change-of-basis matrix to find transformation matrix

## Video transcript

We learned in the last video
that if I have some matrix-- let's call it C, Let's say
it's an n by k matrix. And all of matrix C's columns--
let's say this is column 1, column 2, all the way
through column k-- if each of these columns vectors form
an orthonormal set-- so, let me write this-- columns are
orthonormal set, or the set of columns are an orthonormal set,
we showed in the last video that if I multiply C
transpose times C, I get the identity matrix. I get the k by k identity
matrix. I get the identity matrix of k,
because it's going to be k by n, times n by k. And we saw how everything else
cancels out, except right along the diagonal. And then we were just
multiplying the vectors times themselves, and they're
all unit vectors. So you get the square of their
lengths and you get 1's down the diagonal. We saw that in the last video. Now what happens if
k is equal to n? So what happens if
k is equal to n? Well, then this guy is going
to be a square matrix. And if he's a square matrix,
what else do we know about it? Well, we know that these guys
form an orthonormal set. We saw, I think it was two or
three videos ago, that that means that the columns are
linearly independent. So we also know that we have
linearly independent columns. And we've seen this many,
many times before. If we have a square matrix
with linearly independent columns, it means that
C is invertible. It also means, if we have n of
these, it also means that these columns, or these column
vectors, if they were in a set, then that set would
be a basis for Rn. So I'll just write that on the
side, just because it, you know, it also would mean that,
you know, c1, c2, all the way through cn, because we're saying
n is equal to k, would be a basis for Rn. But we'll leave that on
the side for now. It's just worth pointing out. Now if C is invertible,
what does that mean? That means that there's some C
inverse that exists, such that C inverse times C is equal to
the n by n identity matrix. In this case k and n are equal
to the same thing, so let's just replace that n,
that k with an n. Now these two statements
look very similar. If the columns are an
orthonormal set, and it's an n by n matrix, then we learned
in the last video that C transpose times C is equal to
the n by n identity matrix. And we also know, since it's a
square matrix with linearly independent columns, that
C is invertible. So, in this situation where you
have n by n matrix whose columns form an orthonormal
set, then, well, C inverse must be equal to C transpose. I mean, you can just say, look,
something times C is equal to the identity matrix. Something times C is the
identity matrix. These somethings must
be the same thing. Then C inverse is equal
to C transpose. And that is a huge time saver. If we can assume that these,
this is a square matrix with an orthonormal set. Because finding an inverse
of a matrix is painful, especially if you get,
you know, beyond, you know, n equals 3. If you have to find the inverse
of a, I don't know, 10 by 10 matrix, that would
take you forever. But to find the transpose of a
10 by 10 matrix, a lot easier. You just swap the rows
and the columns. So let's see if we can apply
this newly found insight to simplify some problems that
we've tackled in the past. So I have up here some
vectors, v1, v2, v3. And let's say that we-- let me
copy and paste these so I can use them below, because
I think that might be more useful. Let me copy them and then let
me paste them and put them right down here. We have these vectors here. I'll leave it for you to verify
that they all are unit vectors, and that taken
together they form an orthonormal set, or that
they would be an orthonormal basis for R3. And however you want to view it,
you could verify that on your own time. But what I want to do in this
video is construct an interesting linear
transformation in R3. So let's say I have the plane
formed by v1 and v2. v1 and v2 are going to
be orthogonal to each other as well. This is an orthonormal set. So v1 and v2 are orthogonal. Let's say that this is v1-- and
I'm not drawing what they actually are. I'm just kind of doing a
abstract visual representation of it. v1, v2, they're going
to span some plane. So let me draw the plane
as best as I can. So if you take all of the linear
combinations of v1 and v2, it's going to be
a plane in R3. So that's my plane, and we
know it goes off in every direction infinitely. But this is what
it looks like. We're going to have the
0 vector in it. And let me just call my plane,
this plane, let me call it, it's going to be some subspace V
in R3, which is equal to the span of v1 and v2. Now v3 is orthogonal to
both of these guys. So let me just draw
v3 here for fun. So maybe v3 looks something
like this. It's orthogonal to both of these
vectors, so it's going to be orthogonal to
all of the linear combinations of these guys. So if you view that the plane is
V, then the line spanned by v3, if you imagine it that way,
would be the orthogonal complement of your plane. But let me just draw v3 here.
v3 would be a member of the orthogonal complement, or the
orthogonal complement would be the line spanned by v3. That's all a little
bit of review. Now let's talk about the linear
transformation that I want to construct
in this video. I want to construct a linear
transformation in R3-- remember, we're dealing with
R3 right here-- that essentially reflects any
vector over this plane. So let me draw some indicative
examples. And I hope we can visualize
this well. So let's say I have a vector
that looks like that. So let's call that vector x. And let's say, you know,
let's say it looks something like this. It's above the plane. It kind of jumps out. It's not in my subspace. But I want the transformation
of x to be the mirror image below this. So, you know, if you imagine
that this plane is somewhat transluscent, it would
be down here. My vector x would look
something like this. This would be the transformation
of x that I want to generate. If I have a vector-- let me pick
a color that I haven't used before-- if I have a vector
that looks like this. Then its transformation is going
to go below the plane and just be the mirror image. This is going to be the mirror
image, just like that. You get the idea. Now, what would be, just to
understand our transformation a little bit better,
what would be the transformation of v1? Well v1 is in the plane,
so its mirror image isn't going to change. So the transformation of v1
is just going to be v1. What's the transformation
of v2? Well that's also in the plane. So the transformation of v2
is just going to be v2. And what's the transformation
of v3? Well v3 is directly orthogonal
to the plane. It kind of just pops straight
up out of the plane. So if you want to get its
mirror image, you would literally take the
negative of v3. It would be the negative
of v3. And I do it straight up. We don't know necessarily
whether-- v3 clearly does not go straight up. I just drew it, you know this
plane, I just drew it relative to this plane. This plane might be more tilted
than I've drawn it. But anyway, so the
transformation of v3 will be equal to minus v3. Now this seems like
a fairly tough transformation to construct. You know, we could try to apply
the transformation to our standard basis vectors in
R3 like we've done in the past. But that seems really
complicated, a lot of trigonometry. We would have to figure out the
inclination of this plane, and what we're doing to it. It would just be hard
to visualize. So you probably have a sense
that maybe this transformation will be much easier
to describe if we change our basis. Let's say this is our
standard basis. And our standard basis, our
transformation, we would multiply it times some matrix
x-- sorry, by some matrix A-- to get your transformation
of x. Now, we're saying that this is
hard to find, because it's hard to figure out what the
transformation is when you apply to just your standard
basis vectors. So what I'm going to do is try
to construct some basis, so that when I represent x in that
new basis, if I multiply x times the inverse of our
change of basis matrix, then I'm going to get x in the
new basis coordinates. And maybe it'll be easier to
find the D in this new basis, and then we'll get the B
representation of the transformation of x. And then we can multiply
that times C. So if we're able to figure out
D, what we can do is, look, we know that the transformation
of x is going to be able to A times x. We saw that a long time ago,
that any linear transformation could be represented as a
matrix vector product. But let's say that this
is hard to find. This is hard. Then we can go the other
way around this. We could say that the
transformation of x is equal to you take x, first you
multiply it by C inverse to get the B version of x. So first you multiply
it by C inverse. Then you multiply it by D, to
get the B version of the transformation of x. So then you multiply it by D. And then you multiply that by
C to go back to our standard basis for the transformation. So then you go multiply
it by C. And we've seen this
formula before. So if we can find a nice basis
where D is easy to figure out, then we can multiply it this
way, times the change in basis matrix and its inverse,
and we'll get our A. Because this thing has to be the
same thing as that thing right there. And even more, if we pick an
orthonormal basis, if B is an orthonormal basis, with three
vectors, right, then C will be invertible. Well, we already know if it's an
orthonormal basis, we know that C transpose C is equal
to the identity matrix. And we also know that C is
invertible, if, you know, C is essentially a 3 by 3 matrix. C inverse exists. And in the beginning of this
video, we said, well that means that C transpose is
going to be equal to C Inverse, because this is
just acting like an inverse right there. And then, if C is an n by n
matrix, or in this case we're dealing with a 3 by 3 matrix,
we're dealing with R3, then it simplifies to A will be equal
to C times D, times the C transpose, which is much easier
to find, much easier to find than trying to invert
a 3 by 3 matrix. So let's see if we can
do this effectively. So what would be a good basis? Well I think maybe a natural
one would be to use v1, v2, and v3, because these basis
vectors, it's very easy to figure out what their
transformations are. So let's write that down. So I'm going to make my basis
that I'm going to change to is going to be v1, v2, and v3. So just to make sure we
understand what we're doing, each of these vectors,
what do they look like in my new basis? Well v1 is equal to 1 times
v1, plus 0 times v2, plus 0 times v3. So v1, in my new basis, where v1
is the first basis vector, is just going to be
equal to 1, 0, 0. Same argument. v2, what is it going to be
equal to in my new basis? I don't even have to, I think
you get the idea. Let me just write
it over here. v2 in my new basis is just
going to be 0 times v1-- remember, these numbers, these
coordinates are just the coefficients on my basis
vectors-- it's going to be 0 times v1, plus 1 times
v2, plus 0, times v3. And then finally, v3 in my new
basis is just going to be 0 times v1, plus 0 times v2,
plus 1 times v3, This is almost trivially easy. Now, what is my change of basis
matrix going to be? Well I'm going to have, well,
I'll save that for later. But my change of basis matrix
is just going to be a matrix with these guys as
the columns. And of course its inverse
is going to be the transpose of that. But we'll save that
for a little bit. Now, how can we figure out D? Let's write D. D is going to have
three columns. So d1, d2, d3. It's still a mapping from a
three-dimensional, I guess we could call it a
three-dimensional matrix vector to a three-dimensional
vector. Each of these are
members of R3. So what is the transformation
of v1 represented in B coordinates? That's going to be equal to D
times the representation of v1 in B coordinates, which is equal
to d1, d2, d3 times this guy, right? This is the representation
of v1 in B coordinates. So times 1, 0, 0. Well that just equals d1. We've seen this in the past.
So if we want to figure out what d1 is, it's just the
transformation of v1 in B coordinates. Now we could use that exact same
argument-- let me shift to the left a little bit more. I was running out of
space on the right. So the transformation of v2 in
B coordinates is equal to D, which is d1, d2, d3 times the
B version of v2, or v2 represented in B coordinates. So that is v2 represented in
B coordinates is 0, 1, 0. So that's going to
be equal to d2. And finally, let's
just complete it. The transformation of v3,
represented in B coordinates is going to be equal to d1, d2,
d3 times v3 represented in B coordinates. So times 0, 0, 1. So it's going to be 0 times d1,
plus 0 times d2, plus 1 times d3, is equal to d3. And this is kind of a re-proving
that we can find the columns of D, by essentially
finding the B versions of these
transformations. So D, we can rewrite D is going
to be equal to, with the first column it's just
going to be this. So it's the transformation
of v1 in B coordinates. The second column is this. All right, d2 is that. So it's the transformation
of v2 in B coordinates. And third column is that, the
transformation of v3 in B coordinates. Just like that. So let's see if we
can find these. This, hopefully, shouldn't
be too difficult. So we saw it up here. We wrote what the transformation
of v1, v2, and v3 are. Now we just have to find
out what they are in B coordinates. So let me rewrite, let me copy
and paste this down here, because this might be
nice to look at. So we already found that. I'll paste it down here. So we have that. We've already found that. So we just have to find
the B coordinate representation of them. So the B coordinate
representation of the transformation of v1-- actually,
I think we, well I'll write it out. This becomes a little
bit tedious. That's equivalent to the B
coordinate representation of v1, which is just 1, 0, 0. So let me write D here. D is equal to, the first
column is 1, 0, 0. Now what the second column? Well the transformation of v2,
B representation is going to be just v2's B representation,
or B coordinates, which is equal to 0, 1, 0 zero, 0,
1, 0, and then one left. This one's a little
interesting one. The B representation of the
transformation of v3-- and remember, v3 was the one that's
not in the plane-- that is equal to-- remember, the
transformation is minus v3. Flip that, because we're taking
the mirror image with respect to that plane. So it would be the B
representation of minus v3. Well minus v3 is just 0 times
v1, plus 0 times v2, minus 1 times v3. Those are out basis vectors, so
our third column is going to be 0, 0, minus 1. So we figured out our
D transformation. That was pretty straightforward. D right here is this matrix
that we just figured out. So to figure out A, we can apply
this formula right there with our change of
basis matrix. Actually let me copy
all that stuff. This'll be useful. just let me copy all of this. Copy, then we paste it down
here, now that we have our D. All right, we have all
of this stuff here. And so let's figure
out what A. Is So to do that, first
we have to figure out our change of basis. So I'll just write it
all out like this. Let me clear this out a little
bit, clean things up. All I need is this. I don't need any
of this really. So let me clean all
of that up. So let's figure out what A is. A is equal to our change
of basis matrix. Well our change of basis matrix
is just the matrix with these guys as the columns. Well let me just take
out the 1/3. So C is 1/3, times 2 minus 2,
1, 2, 1 minus 2, and then we have 1, 2, 2, right? That right there is A, and then
we're going to multiply that times-- or this right here
is our change of basis matrix, that is C-- and we're
going to multiply that times D, which is 1, 0, 0, 0, 1, 0,
and then 0, 0, minus 1. So it almost looks like the
identity matrix, but we flipped our third vector,
and that's why we got a minus 1 there. And then we have C inverse,
but because C was a square matrix with orthonormal columns,
we know that C inverse is the same thing
is C transpose. So let me write that here. So let me write this here. So it's just going to be the
transpose of this thing. So I'll write another 1/3 out
here, just to simplify things. So times 1/3, times the
transpose of this character, so we have 2, 2, 1, so we're
going to have 2, 2, 1. We have minus 2, 1, 2, so
we have minus 2, 1, 2. We have 1 minus 2, 2, so we
have 1 minus 2 and 2. And this right here, this is C
inverse, which is equal to C transpose, because C is an
invertible matrix, or a square matrix, with orthonormal
columns. So what's this going
to be equal to? Well let's just take, I don't
know, first let's just take-- let me write this, this was D--
so let's just take this product first. We'll worry
about the 1/3's later. And now I can erase this. I can use this real estate. Let me erase that. So let me turn my pen
tool back on. All right. So let's take this product
right there. So A is equal to, let's take
the 1/3's, it's going to be equal to 1/9 times-- now
this is going to be another 3 by 3 matrix. So, we're going to have
2, 2, 1, times 1, 0 0. Or we're going to dot 2,
2, 1 with 1, 0, 0. So the only term that's going
to be non-zero is the 2 times the 1. So that's going to be 2. Then we're going to dot 2, 2, 1
with 0, 1, 0, right, for the second column. The only non-zero term
is going to be the 2, the middle 2. Then you get 2, 2, 1
dot 0, 0, minus 1. So that's going to be, the only
non-zero term is this last one which is going
to be negated, so it's going to be minus 1. This isn't too bad. I simplified it, but this is
almost the identity matrix. Then we have this guy. We have minus 2, 1, 2 dotted
this guy, only this guy kind of survived that dot product,
minus 2, then you take this guy dot this guy, only
this guy survives. And then you take this
guy dot that guy. This guy gets negative. And he's the only survivor. I think you see what's
happening. These rows stay the same,
but the third term becomes negative. Let's do it again. So you take this guy. You take the dot product of this
column, this row vector with this column vector. Only this guy survives. Then this row with this column,
you have the minus 2. And then this row with that
column, only the 2 survives, but it's times minus 1, so
it becomes a minus 2. And then we have this
vector out here. Remember, we took 1/3's,
we multiplied them and we got 1/9. And then we have 2 minus
2, 1, 2, 1, minus 2, and then 1, 2, 2. And this'll be a little bit more
painful to multiply, but I think we can pull it off. Because this is the
home stretch. So A, our transformation matrix
for that, you know, strange transformation where
we're reflecting over the plane in R3 is going to be
equal to 1/9 times-- it's going to be a 3 by 3 matrix. So it's going to be 2
times 2, which is 4. Let me write this down. Oh, that's going to
be too onerous. So 2 times 2, which is 4, plus
2 times 2, which is 4, and then plus minus 1, right? So it's going to be 4 plus
4, minus 1, which is 7. 4 plus 4 minus 1 is 7. So we're going to multiply this
guy times this column. So 2 times minus 2, is minus
4, plus 2, is minus 2. Minus 2 is minus 4. Then we're going to have 2 times
1 is 2, minus 4, which is minus 2, and then minus
2, which is minus 4. Not so painful. OK, now in the second row, we're
going to have minus 2 times minus 2, is minus 4,
plus 2, which is minus 2, minus 2, which is minus 4. Then we're going to have minus
2 times minus 2, which is 4, plus 1, which is 5, minus 4. Let me make sure I
got that right. We're going to have minus 2
times minus 2, which is positive 4, plus 1, times 1,
which is minus 3, plus minus 2 times minus 2, which is
which is minus 4. And so we are going to get-- and
I'm confusing myself-- I'm going to get-- oh no,
this thing is going to get very confusing. It's straightforward. Minus 2 times minus 2 is 4,
plus 1 is 5, minus 4 is 1. And then we get minus 2 minus 2,
which is minus 4, minus 4, which is minus 8. Home stretch. This is the painful part. OK, then you have 2 minus 4,
which is minus 2, minus 2, which is minus 4, minus 4-- all
right, now-- minus 2 minus 2, minus 4, and that's
minus 8. And then finally you have 1 plus
4, minus 4, which is 1. And if I haven't made any
careless mistakes, we're done. We now know that our
transformation that reflects-- now that did all this fancy
stuff-- our transformation that reflects across this plane
that's spanned by these characters right there, it can
be represented by this matrix. So our transformation, applied
to x in standard coordinates, is just equal to A times
x, where A is equal to this matrix. We can multiply the 1/9 out,
but that will just make it look more confusing. But this would've been really
hard to figure out on our own, to figure out that we had to
put a 7 and a minus 4 or to even apply the transformation to
the standard basis vectors, which you normally do to
figure out this matrix. Instead, we changed our basis
to kind of a very natural orthonormal basis. And the fact that it was
orthonormal made it very easy to find the inverse of our
change-of-basis matrix. Anyway, hopefully you
found that useful.