Main content

## Optimizing multivariable functions

Current time:0:00Total duration:8:04

# Multivariable maxima and minima

## Video transcript

- [Voiceover] When you have
a multivariable function, something that takes in
multiple different input values and let's say it's just
outputting a single number, a very common thing you wanna do with an animal like this is Maximize it. Maximize it, and what this means is you're looking for the input points, the values of x and y and
all of its other inputs, such that the output, f, is as
great as it possibly can be. Now this actually comes up
all the time in practice 'cause usually when you're dealing with a multivariable function,
it's not just for fun and for dealing with abstract symbols, it's 'cause it actually
represents something, so maybe it represents
profits of a company, maybe this is a function
where you're considering all the choices you can make, like the wages you give your employees or the prices of your goods, or the amount of debt that
you raise for capital, all sorts of choices that you might make, and you wanna know what
values should you give to those choices such
that you maximize profits, you maximize the thing,
and if you have a function that models these relationships, there are techniques, which
I'm about to teach you, that you can use to maximize this. Another very common setting, more and more important these days, is that of machine learning
and artificial intelligence, where often what you do
is you assign something called a cost function to a task, so maybe you're trying to teach a computer how to understand audio or
how to read handwritten text. What you do, is you find a function that basically tells it how wrong it is when it makes a guess,
and if you do a good job designing that function, you
just need to tell the computer to Minimize, so that's kind
of the flip side, right? Instead of finding the maximum, to minimize a certain function, and if it minimizes this cost function, that means that it's
doing a really good job at whatever task you've assigned it, so a lot of the art and
science of machine learning and artificial intelligence
comes down to, well, one finding this cost function and actually describing difficult tasks in terms of a function, but
then applying the techniques that I'm about to teach you to have the computer minimize that, and a lot of time and research has gone into figuring out
ways to basically apply these techniques, but really
quickly and efficiently. So, first of all, on a conceptual level, let's just think about what it means to be finding the maximum
of a multivariable function. So I have here the graph
of a two-variable function. It's something that has
a two-variable input that we're thinking of as the xy-plane, and then its output is
the height of this graph, and if you're looking to maximize it, basically, what you're
finding is this peak, kind of the tallest
mountain in the entire area, and you're looking for the input value, the point on the xy-plane
directly below that peak, 'cause that tells you
the values of the inputs that you should put in to
maximize your function, so how do you go about finding that? Well, this is perhaps the core observation in well, calculus, not just
multivariable calculus. This is similar in the
single variable world, and there are similarities
in other settings, but the core observation is that if you take a
tangent plane at that peak, so let's just draw in a
tangent plane at that peak, it's gonna be completely flat, but let's say you did this
at a different point, right? 'Cause if you tried to
find the tangent plane, not at that point, but you
kind of moved it about a bit to somewhere that's not quite a maximum, if the tangent plane has
any kind of slope to it, what that's telling
you is that if you take very small directions, kind of in the direction
of that upward slope, you can increase the
value of your function, so if there's any slope
to the tangent plane, you know that you can walk in
some direction to increase it, but if there's no slope
to it, if it's flat, then that's a sign that no
matter which direction you walk, you're not gonna be significantly
increasing the value of your function. So what does this mean
in terms of formulas? Well, if you kind of think back to how we compute tangent planes and if you're not very
comfortable with that, now would be a good time
to take another look at those videos about tangent planes, the slope of the plane in each direction, so this would be the
slope in the x direction, and then if you look at it
from another perspective, this would be the slope
in the y direction, each one of those has
to be zero, and that, in terms of partial derivatives, means the partial
derivative of your function, at whatever point you're
dealing with, right? So I'll call it x not, y not, as the point where you're inputting
this, has to be zero, and then similarly, the partial derivative with respect to the other
variable, with respect to y, at that same point, has to be zero, and both of these have to be true because let's just take
a look, I don't know, let's slide it over a little bit here, this tangent plane, if
you look at the slope, you imagine walking in the y direction, you're not increasing your value at all. The slope in the y direction
would actually be zero, so that would mean the partial derivative with respect to y would be
zero, but with respect to x, when you're moving in
the x direction here, the slope is clearly negative, because as you take positive
steps in the x direction, the height of your tangent
plane is decreasing, which corresponds to if you
take tiny steps on your graph, then the height will decrease
in a manner proportional to the size of those tiny steps. So what this gives you
here is gonna be a system of equations where you're
solving for the value of x not and y not that satisfies
both of these equations, and in future videos, I'll go through specific examples of this. For now, I just wanna give a
good conceptual understanding, but one very important thing to notice is that just because this
condition is satisfied, meaning your tangent plane is flat, just because that's satisfied,
doesn't necessarily mean that you've found the maximum. That's just one requirement
that it has to satisfy, but for one thing, if you
found the tangent plane at other little peaks, like
this guy here or this guy here, or all of the little bumps that go up, those tangent planes would also be flat, and those little bumps
actually have a name because this comes up a lot. They're called local Minima,
or local Maxima, sorry, so those guys are called local Maxima. Maxima is just the plural of Maximum, and local means that it's
relative to a single point, so it's basically, if you
walk in any direction, when you're on that little
peak, you'll go downhill, so relative to the neighbors
of that little point, it is a maximum, but relative
to the entire function, these guys are the shorter
mountains next to Mount Everest, but there's also another circumstance where you might find a flat tangent plane, and that's at the Minima points, right? If you have the global
Minimum, the absolute smallest, or also just the local
Minima, these inverted peaks, you'll also find flat tangent planes. So what that means, first of all, is that when you're minimizing a function, you also have to look
for this requirement, where all the partial
derivatives are zero, but it mainly just means
that your job isn't done once you've done this. You have to do more tests to check whether or not what you
found is a local Maximum or a local Minimum, or a global Maximum, and these requirements, by the way, often you'll see them written
in a more succinct form, where instead of saying
all the partial derivatives have to be zero, which
is what you need to find, they'll write it in a different form where you say that the
gradient of your function, f, which, of course, is just the vector that contains all those
partial derivatives. Its first component is
the partial derivative with respect to the first variable, its second component is
the partial derivative with respect to the second variable, and if there's more variables,
you would keep going, you'd say that this whole thing has to equal the zero vector,
the vector that has nothing but zeroes as its components, and it's kind of a
common, abusive notation. People will just call
that zero vector, zero, and maybe they'll emphasize
it by making it bold, because the number zero is not a vector and often making things bold emphasizes that you want to be referring to a vector, but this gives a very succinct way of describing the requirement. You're just looking for where
the gradient of your function is equal to the zero vector, and that way, you can just write it on
one line, but in practice, every time that you're expanding that out, what that means is you find all of the different partial derivatives, so this is really just a matter
of notational convenience and using less space on a blackboard, but whenever you see this,
that the gradient equals zero, what you should be thinking of is the idea that the tangent plane, the tangent plane is completely flat, and as I just said, that's not enough because you might also
be finding local Maxima or Minima points, but in
multivariable calculus, there's also another possibility, a place where the tangent plane is flat, but what you're looking at
is neither a local Maximum nor a local Minimum, and this is the idea of a saddle point, which is
new to multivariable calculus, and that's what I'll be talking
about in the next video, so I will see you then.