If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

Main content
Current time:0:00Total duration:10:32

Video transcript

so far when I've talked about the gradient of a function and you know let's think about this as a multivariable function with just two inputs those are the easiest to think about so maybe it's something like x squared plus y squared very friendly function what I've talked about the gradient I've left open a mystery we have the way of computing it and the way that you think about computing it is you just take this vector and you just throw the partial derivatives in there partial with respect to X and the partial with respect to Y with respect to Y and if it was a higher dimensional input then the output would have as many as many variables as you need if it was F of XYZ you'd have partial X partial Y partial Z and this is the way to compute it but then I gave you a graphical intuition I said that it points in the direction of steepest descent and maybe the way you think about that is you have your input space which in this case is the xy-plane and you you think of it as somehow mapping over to the number line do you have put space and if you have a given point somewhere the question is of all the possible directions that you can move away from this point all those different directions you can go which one of them you know the like this point will land somewhere on the function and as you move in the various directions maybe one of them not just your output a little bit one of them nudges a lot one of it slides at negative you know one of them slides at negative a lot which one of these directions results in the greatest increase to your function and this was the loose intuition if you want to think in terms of graphs we could look over at the graph of f of x squared and this is the gradient field all of these vectors in the xy-plane are the gradients and you kind of look from below you can maybe see why each one of these points in the direction you should move to walk uphill on that graph as fast as you can and if you're a mountain climber and you want to get to the top as quickly as possible these tell you the direction that you should move to go as quickly this is why you call it direction of steepest ascent so back over here I don't see the connection immediately or at least when I when I was first learning about it it wasn't clear why this this combination of partial derivatives has anything to do with choosing the best direction and now that we've learned about the directional derivative I can give you a little bit of an intuition so let's say instead of thinking about you know all the possible directions and all of the possible changes to the output that they have so all fill in my line there you know let's say you just have you've got your point or you're evaluating things and then you just have a single vector now let's actually make it a unit vector mate let's make it make it the case that this guy has a length of one so I'll go over here and I'll just think of that guys being V and say the V has a length of one so this is this is our vector we know now having learned about the directional derivative that you can tell the rate at which the function changes as you move in this direction by taking the directional derivative of your function and let's say this point I don't know what's a good name for this point just like a B a B at this point when you evaluate this at a B and the way that you do that is just dotting the gradient of F I should say dotting it evaluated at that point because gradient is a vector valued function and we just want a specific vector here so evaluating that at your point a B together with whatever the vector is whatever that whatever that value is and in this case we're thinking of V as a unit vector so this this is how you tell the rate of change and when I originally introduced the directional derivative I gave kind of a Kevin and kind of an indication why you know if you imagine dotting this together with and on let's say it was a vector that's like 1 2 really you're thinking this vector represents one step in the x direction two steps in the Y direction so the amount of their changes things should be 1 times the change caused by a pure step in the X Direction plus 2 times a change caused by a pure step in the Y direction so that was kind of the loose intuition you can see the directional derivative video if you want a little bit more discussion on that and this is this is the formula that you have but this starts to give us the key for how we could choose the direction of steepest descent because now what we're really asking we say which one of these changes things the most you know maybe when you move in that direction it changes f you know a little bit negatively and we want to know you know maybe does another vector W is the change caused by that going to be positive is it going to be as big as possible what we're doing is we're saying find the maximum for all unit vectors so for all vectors V that satisfy the property that their length is 1 find the maximum of the dot product between F evaluated at that point right evaluated at whatever point we care about and V find that maximum well let's just think about what the dot product represents so let's say we go over here and let's say you know let's say we evaluate the gradient vector and it turns out that the gradient points in this direction and maybe it's you know it doesn't have to be a unit vector it might be something very long like that so if you imagine some vector V you know some unit vector V let's say it was sticking off in this direction the way that you interpret this dot product the dot product between the gradient F and this new vector V is you would project that vector directly kind of a perpendicular projection onto your gradient vector and you'd say what's that length you know what's that length right there and just as an example it would be something a little bit less than one right because this is a unit vector so as an example let's say that was like 0.7 and then you'd multiply that by the length of the gradient itself of that of the vector against which your dotting and maybe and maybe that guy maybe the length of the entire gradient vector just again as an example maybe that's two doesn't have to be it could be anything but the way that you interpret this whole dot product then is to take the product of those two you would take 0.7 the length of your projection times the length of the original vector and the question is when is this maximized what unit vector maximizes this and if you start to imagine maybe swinging that unit vector around so you know if instead of that guy you were to use one that point it a little bit more closely in the direction then it's projection would be a little bit longer maybe that projection would be like 0.75 or something if you take the unit vector that points directly in the same direction as that full vector then the length of its projection is just the length of the vector itself it would be 1 because projecting it doesn't change what it is at all so it shouldn't be too hard to convince yourself and if you have shaky intuitions on the dot product I'd suggest finding the videos we have on Khan Academy for those Sal does a great job giving giving that deep intuition the it should it should kind of make sense why the vector that points the unit vector the points in the same direction as your gradient is going to be what maximizes it so the answer here the answer to what vector maximizes this is going to be well it's it's the gradient itself right it is that gradient vector you know evaluated at the point we care about except you'd normalize it right because we're only considering unit vectors so to do that you just divide it by whatever its magnitude is if its magnitude was already 1 it stays 1 if its magnitude was 2 you're dividing it down by 1/2 so this is your answer this is the direction of steepest descent so I think one thing to notice here is the most fundamental fact is that the gradient is this tool for computing directional derivatives you can think of that vector is something that you really want to dot against and that's actually a pretty a pretty powerful thought is that the gradient it's not just a vector it's a vector that loves to be dotted together with other things that's the fundamental and as a consequence of this as a consequence of that the direction of steepest descent is is that vector itself because anything if you're saying what maximizes the dot product with that thing it's well the vector the points in the same direction as that thing and this can also give us an interpretation for the length of the of the gradient we know the direction is the direction of steepest descent but what is the grid the lengths mean so let's give this guy a name let's give this normalized version of it a name I'm just going to call it W so W will be the unit vector that points in the direction of the of the gradient if you take the directional derivative in the direction of W of F what that means is the gradient of F dotted with that W and you kind of spell out what what W means here that means you're taking the gradient of the vector dotted with itself but because it's W and not not the gradient we're normalizing or dividing that not by nine - Dov EV that doesn't really make sense but by the value of the gradient and all of these I'm just writing gradient of F but maybe you should be thinking about gradient of F evaluated at a B but I'm just being kind of lazy and just just writing gradient of F and the top when you take the dot product with itself what that means is the square of its magnitude but the whole thing is divided by the magnitude so you can kind of cancel that out you could say this doesn't need to be there that exponent doesn't need to be there and basically the directional derivative the directional derivative in the direction of the gradient itself has a value equal to the magnitude of the gradient so this tells you one when you're moving in that direction in the direction of the gradient the rate at which the function changes is given by the magnitude of the gradient so this is this really magical vector it does a lot of things it it's the tool that lets you dot against other vectors to tell you the directional derivative as a consequence it's the direction of steepest descent and it's magnitude tells you the rate at which things change while you're moving in that direction of steepest ascent it's just really am a core part of scalar valued multivariable functions and it is it is the extension of the derivative in in every sense that you could want a derivative to extend