- Introduction to orthonormal bases
- Coordinates with respect to orthonormal bases
- Projections onto subspaces with orthonormal bases
- Finding projection onto subspace with orthonormal basis example
- Example using orthogonal change-of-basis matrix to find transformation matrix
- Orthogonal matrices preserve angles and lengths
- The Gram-Schmidt process
- Gram-Schmidt process example
- Gram-Schmidt example with 3 basis vectors
Showing that orthogonal matrices preserve angles and lengths. Created by Sal Khan.
Want to join the conversation?
- Should Sal be saying orthonormal the whole time instead of orthogonal? Because he is utilizing C^T = C^ -1 ( C transpose = C inverse) in his proofs and I believe he previously proved this for orthonormal change of basis matrices not orthogonal change of basis matrices.
Thank you and please correct me if I am wrong.(16 votes)
- Sal says in the beginning of the video that the name for an orthonormal set in matrix form is an orthogonal matrix... My textbook calls them orthonormal matrices, which seems a lot less confusing to me.(20 votes)
- Is C inverse, or C transpose, also an orthonormal matrix? Thanks!(5 votes)
- Straightforward from the definition: a matrix is orthogonal iff tps(A) = inv(A). Now, tps(tps(A)) = A and tps(inv(A)) = inv(tps(A)). This proves the claim. You can also prove that orthogonal matrices are closed under multiplication (the multiplication of two orthogonal matrices is also orthogonal): tps(AB) = tps(B)tps(A)=inv(B)inv(A)=inv(AB). Hope this helps :)(3 votes)
- does adding two nxn orthogonal matrices result in an nxn orthogonal matrix(2 votes)
- Good question,
To answer a question like this you should first try some examples. The best examples are easy examples. So let's try some 1x1 matrices. There are only two orthogonal matrices given by (1) and (-1) so lets try adding (1) + (1)=(2). (2) is not orthogonal so we have found a counterexample!.
In general you will see that adding to orthogonal matrices you will never get another since if each column is a unit vector the sum of two unit vectors cannot be a unit vector.(6 votes)
- You show cos(teta) = cos (teta_t), but i think it does not prove teta = teta_t, because teta_t also can be 360-teta, i.e., teta = -teta_t. Am i wrong or it does not matter?(3 votes)
- At4:48, in order to prove that the length of a vector does not change when represented with respect to orthogonal basis, shouldn't we start by finding length of C^-1*X rather than C*X because X represented in basis B is equal to C^-1*X ?(2 votes)
- He's using C here to represent any transformation with matrices whose columns are orthonormal basis vectors. Its not important here that it can transform from some basis B to standard basis.
We know that the matrix C that transforms from an orthonormal non standard basis B to standard coordinates is orthonormal, because its column vectors are the vectors of B. But since C^-1 = C^t, we don't yet know if C^-1 is orthonormal. All we know is that its r o w vectors are an orthonormal set.(1 vote)
- At4:03Sal mentions that orthonormal matrices "rotate [vectors] around." In the last video we saw an example of an orthonormal matrix reflecting vectors. Are there any other geometric transformations that an orthonormal matrix could produce?(1 vote)
- No, those are the only two (though there can be combinations of rotation and reflection). Orthonormal matrices are special in that they preserve the relative orientation and size of vectors, so they will never scale a vector.(1 vote)
- Previous videos are referred to, but I am unaware of the order of the videos. If I knew what video was prior and this video is confusing then I feel I may need to watch the preceding videos. I don't know the order.(1 vote)
- You can access all the titles in order by just looking on the left, also by clicking on the titles at the upper left.(1 vote)
- Since the angle between two vectors is preserved under reflections (when dealing with orthogonal transformation matrices, at least), how does one go about defining directed angles in R^n?(1 vote)
- My guess is that in R^2 use delta(arctan(x2/x1)) between the two vectors. For R^3 and greater, which direction is positive?(1 vote)
- Just because the cosine of the angle is preserved doesn't necessarily imply the angle is as well. What could be the case is that one of the vectors in question has now the opposite direction, and then, in turn, we have the angle being 180-theta, as by definition of angle between vectors we look at the angle in front of the (a-b) side. So should we instead be talking about the cosine being preserved rather than the angle??(1 vote)
In the last couple of videos, we've seen that if we have some matrix C that is n by n. It's a square matrix, and is columns, column form and orthonormal set. Which just means that the columns each have been normalized. So they each have length of 1 if you view them as column vectors. And they're all mutually orthogonal to each other. So if you dot it with yourself you get 1. If you dot it with any of the other columns, you get 0. We've seen this multiple times. It's orthogonal to everything else. If you have a matrix like this-- and I actually forgot to tell you the name of this-- this is called an orthogonal matrix. We've already seen that the transpose of this matrix is the same thing as the inverse of this matrix. Which makes it super, duper, duper useful to deal with. The transpose of this matrix is equal to the inverse. Now, this statement leads to some other interesting things about this. So, so far we've been dealing this mainly with the change of basis. I can kind of draw the diagram that you're probably tired of by now. Let's say that's the standard basis. Let's say that I have x in coordinates with another basis. We've seen I can multiply this guy times c. To get that up there I could multiply that guy by c inverse to get this guy right here. And, in that world, we viewed c as just a change of basis. Were representing the same matrix-- we're representing the same vector. We're just changing the coordinates of how we represent it. But we also know that any matrix product, any matrix vector product, is also a linear transformation. So, this change of basis is really just a linear transformation. What I want to show you in this video, and you could view it either as a change of basis or as a linear transformation, is that when you multiply this orthogonal matrix times some vector, it preserves-- let me write this down-- lengths and angles. So let's have a little touchy-feely discussion of what that means. Let's view it as a transformation. Let's say I have some set of vectors in my domain. Let's say they look like this. Let's say that it looks like this. Well, let me do it like-- I'll draw that one like that guy, and this guy like that. And there's some angle between them. Angles are easy to visualize in r2, r3. Maybe a little harder once we get the higher dimensions. But that's the angle between them. Now, if we're saying that we're preserving the angles and the lengths, that means if I were to multiply these vectors times c then we could view it as a transformation. Maybe I rotate them or I-- well, you can't really. Maybe I rotate them or do something like that. So maybe that pink vector will now look like this. But it's going to have the same length. This length is going to be the same thing as that length. And even more, when I said it preserves lengths and angles, this yellow vector's going to look something like this. Where the angle is going to be the same. Where this data is going to be that data. That's what I mean by preserves angles. If we didn't have this case, we could imagine a transformation that doesn't preserve angles. Let me draw one that doesn't. If this got transformed to, I don't know, let's say this guy got a lot longer, and let's say this guy also got longer, and I want to show that the angle also doesn't get preserved. Not only did it get longer, but it got distorted a little bit. So, the angle also changed. This transformation right there is not preserving angles. So when you have a change of basis matrix that's orthogonal, when you have a transformation matrix that's orthogonal, all it's essentially doing to your to your vectors, is it kind of a rotates them around, but it's not going to really distort them. So I'll write that in quotes because that's not a mathematically rigorous term. So, no distortion of vectors. So, I've kind of showed you the intuition of what that means. Let's actually prove it to ourselves that this is the case. So, I'm saying that if this pink vector here is x, and that this pink vector here is c times x, I'm claiming that the length of x is equal to the length of c times x. Let's see if that's actually the case. The length of cx squared is the same thing as cx dot cx. And here it's always useful for me to kind of remind myself that if I take two vectors-- let me do it over here. Let's say I have y dot y. This is the same thing as y transpose, if you view them as matrices, y transpose times y. y transpose y is just y1, y2, all the way to yn times y1, y2, all the way to yn. And if you were to do this 1 by n times n by 1 matrix product, you're going to get a 1 by 1 matrix or just a number that's going to be y1 times y1 plus y2 times y2 all the way to yn times yn. So, this is the same thing as y dot y. I think I did this about ten or twenty videos ago, but it's always a good refresher. So let's use this property right here. So these two dotted with each other. This is the same thing is taking one of their transpose times the other one. So turn this from a vector, vector dot product to a matrix, matrix product. So this is the same thing as CX transpose, CX. so you can view this as a 1 by n matrix now, times the 1 by 1 matrix which is just the column vector cx. These are the same thing. Now, we also know that A times B transpose is the same thing is B transpose, A transpose. We saw that a long time ago. So this thing right here is going to be equal to X transpose, C transpose. Just switch the order and take the transpose of each. X transpose times C transpose. And then you have that times CX. And now we know that C transpose is the same thing is as C inverse. This is where we need the orthogonality of the matrix C. This is where we need it to be a square matrix where all of its columns are mutually orthogonal and they're all normal. And so this thing is just going to become the identity matrix. I can write the identity matrix there, but that's just going to disappear. So this is going to be equal to X transpose X. X transpose is the same thing as X dot X which is the same thing as the length of X squared. So the length of CX squared is the same thing as the length of X squared. So, that tells us that the length of X, or the length of CX, is the length of x because both of these are going to be positive quantities. So I've shown you that orthogonal matrices definitely preserve length. Let's see if they preserve angles. So we actually have to define angles. Throughout our mathematical careers, we understood what angles mean in kind of r2 or r3. But in linear algebra, we like to be general. And we defined an angle using the dot product. We use the law of cosines and we took an analogy to kind of triangle in r2. But we defined an angle or we said the dot product V dot W is equal to the lengths, the products of the lengths of those two vectors times the cosine of the angle between them. Or you could say that the cosine of the angle between two vectors, we defined as the dot product of those two vectors divided by the lengths of those two vectors. This was the definition so that we can extend the idea of an angle to an arbitrarily high dimension to r google if we had to. So let's see if it preserves. Let's see what the angle is if we multiply these guys by C. So, if we wanted-- let's say our new angle. So, cosine of angle C. Once we perform our transformation. We're going to perform the transformation on all of these characters. It's going to be CV dot CW over the lengths of CV times the lengths of CW. Now we already know that lengths are preserved. We already know that the length of CW and CV are just going to be W and V. We just proved that. Let me write that. So the cosine of theta C is equal to CV dot CW over the lengths of V times W. Because we've already shown that it preserves lengths. We'll see what this top part equals. So we can just use the general property. The dot product is equal to the transpose of one guy as kind of a matrix times the second guy. So this is equal to CW transpose times CV. And all of that over these lengths. Like the W. And this is going to be equal to-- I'm going to write it down here to have some space. We can switch these guys and take their transpose. So, it's W transpose times C transpose times CV. All of that over their lengths, the product of their lengths V and W. And this is the identity matrix. That's the identity matrix, and this is going to be equal to W transpose times V over the products of their lengths. And this is the same thing as V dot W. This is V dot W over their lengths. Which is cosine of theta. So, you notice, by our definition of an angle as the dot product divided by the vector lengths, when you perform a transformation or you can imagine a change of basis either way, with an orthogonal matrix C the angle between the transformed vectors does not change. It is the same as the angle between the vectors before they were transformed. Which is a really neat thing to know. The change of bases or transformations with orthogonal matrices don't distort the vectors. They might just kind of rotate them around or shift them a little bit, but it doesn't change the angles between them.