Main content
Course: Multivariable calculus > Unit 3
Lesson 5: Lagrange multipliers and constrained optimization- Constrained optimization introduction
- Lagrange multipliers, using tangency to solve constrained optimization
- Finishing the intro lagrange multiplier example
- Lagrange multiplier example, part 1
- Lagrange multiplier example, part 2
- The Lagrangian
- Meaning of the Lagrange multiplier
- Proof for the meaning of Lagrange multipliers
© 2024 Khan AcademyTerms of usePrivacy PolicyCookie Notice
Constrained optimization introduction
The Lagrange multiplier technique is how we take advantage of the observation made in the last video, that the solution to a constrained optimization problem occurs when the contour lines of the function being maximized are tangent to the constraint curve. Created by Grant Sanderson.
Want to join the conversation?
Hi Grant,
Why are you assuming the maximum value occurs when contour line is tangent to constraint?
What if the contour line which passes through the origin and intersects constraint at two different point happens to be the maximum value for that function?(16 votes)
According to the math stack exchange its just an intuition:
https://math.stackexchange.com/questions/2578903/lagrange-multipliers-tangency(1 vote)
What program do you use to model functions? Sometimes I need to model a function and cannot picture what it looks like, and cannot find anything online to help me picture it. So, I'd like to maybe get a program like the one you use or, if possible, maybe even the exact same program.(3 votes)
It is called 'grapher'. It is standard software that comes with mac osx on Apple computers.(14 votes)
I don't feel this explains the essence of Lagrange multipliers. You have to say why the gradient of f is a multiple of gradient g. The reason is that when f(x,y) is constrained to the curve/surface g(x,y), we need to find a point (a,b) such that grad(f(a,b)) is perpendicular to this curve/surface g(x,y) which is a candidate for an extrema. To see why extrema of f must have gradients that are perpendicular to the curve/surface g, consider a non-perpedicular grad(f) - it can't possibly be a max or min because it has components pointing in the directions of the surface. Since gradients point in directions of max increase, a max or min should not point in any directions along the curve/surface, because if they do clearly there is something larger/smaller immediately next to it. So grad(f) must be perpendicular, otherwise if it has components along the surface you just follow them until you reach a point that doesn't budge.
Finally, all grad(g) are normal to g since they are level curves to g = 0. So finding the normal gradients of f is simplified since they must point in the same direction of grad(g). That is why they are scalar multiples of each other.
This video elaborates on this idea: https://www.youtube.com/watch?v=WOw7TWueU_A(3 votes)- what if we want to find the minimum value?(1 vote)
- Finding the minimum value follows the exact same process as finding the maximum except that after finding the critical points, you chose the point with the lowest value instead of the largest.(4 votes)
Interesting.
I imagine this this would involve solving that tangency point. That kind of implies though that your constraint has to be a smooth curve? i.e. differentiable? i.e. I can't have a squared constraint right?
I imagine you can represent square waves using a fourier series but I wonder how it would be possible to represent a square as a differentiable function.(2 votes)
You could take the min/ max on each of 4 straight lines within bounds (i.e., on line segments) as the constraints and compare them to find the min min or max max.(1 vote)
- why maximum values occurs tangent to the unit circle?
please really need answer.(2 votes) - Hi Grant,
I have one question, What software do you use to plot that amazing graphs?(2 votes) 
Why can't we just solve for x^2 in terms of y, plug it into the equation, and then solve it like a normal single variable calculus problem?(2 votes)
This only works assuming we know the constant value of f(x,y) = c such that is tangent to the unit circle. In order to make the substitution you suggest, it requires knowing this value of c, so you cannot approach it this way.(0 votes)
- https://www.math3d.org/iuCqFiLU
The maximum of this function along x^2+y^2=5/3 does not occur along a contour line tangent to the circle. How does this mesh with what Grant is saying about constrained optimization?(1 vote)- Your circle meets your function's surface at ridges of contant height (so the top of the ridge is a contour line marking the maximum height of the ridge) where the gradient of your surface is 0. Clearly, these are special cases, but in general (it seems to me) - and where ∇f ≠ 0 - the constrained maximum or minimum will be where the contours are tangential to the constraining circle.(1 vote)
Hi Grant,
Why would we use Lagrangians instead of pluging in the constraits?
Namely, why max f(x, y) = x^2*y subject to x^2+y^2=1
is different than max f(y) = (1-y^2)*y ?
For both the solution is the same: y=1/root(3)(1 vote)
You bring up a good point. However the functions you have just work out nicely, and they don't always do this (try, for example, f(x,y)=xy, 4x^2+9y^2=32, or f(x,y)=x^2y+x+y, xy=4). Also remember that your answer will have two points, from the positive or negative square root from the constraint. Although that root gets squared and the y-value is always positive, you still will have a negative root and a positive root for the x-value.
From what I can tell, Lagrangians were developed because someone noticed that if you were to draw a constraint on a 3D graph and look for maximums or minimums, the gradients would point in the same direction, but the magnitudes would be different by a factor. We call this factor the Lagrange multiplier.
Check out these resources for more help visualizing this idea, which can be super helpful and important.
1) Look up "Understanding Lagrange Multipliers Visually" by Serpentine Integral on YouTube.
2) Look up "Lagrange Multipliers | Geometric Meaning & Full Example" by Dr. Trefor Bazett on YouTube.
3) https://www.geogebra.org/m/gCWD3M4G. This is an interactive program for visualizing gradients and lagranges.(1 vote)
Video transcript
- [Instructor] Hey everyone, so in the next couple of videos, I'm going to be talking about a different sort of optimization problem, something called a Constrained
Optimization problem, and an example of this is
something where you might see, you might be asked to maximize some kind of multi-variable function, and let's just say it was the function f of x,y is equal to x squared, times y. But that's not all you're asked to do, you're subject to a certain constraint where you're only allowed values of x and y on a certain set, and I'm just going to say, the set of all values of x and y, such that, x squared plus
y squared, equals one. And this is something you might recognize as the unit circle, this particular constraint
that I've put on here, this is the unit circle. So one way that you might think
about a problem like this, you know, you're maximizing a
certain two-variable function, is to first think of the
graph of that function. That's what I have pictured here, is the graph of f of x,y,
equals x squared, times y. And now this constraint,
x squared plus y squared, is basically just a
subset of the x,y plane. So if we look at it head on here, and we look at the x,y plane, this circle represents all of the points x,y, such that, this holds. And what I've actually drawn here isn't the circle on the x,y plane, but I've projected it up onto the graph. So this is showing you
basically the values where this constraint holds, and also what they look like when graphed. So the way you can think
about a problem like this, is that you're looking on this circle, this kind of projected
circle onto the graph, and looking for the highest points. And you might notice kind of here, there's sort of a peak
on that wiggly circle, and over here there's another one, and then the low points would be, you know, around that
point and around over here. Now this is good, and I think this a nice way to sort of wrap your head around
what this problem is asking, but there's actually a
better way to visualize it, in terms of finding the actual solution, and that's to look only in the x,y plane, rather than trying to graph things, and just limit our perspective
to the input space. So what I have here are the contour lines for f of x,y equals x
squared plus y squared. And if you're unfamiliar with contour lines or a contour map, I have a video on that, you can go back and take a look, it's going to be pretty crucial for the next couple videos
to have a feel for that. But basically, each one
of these lines represents a certain constant value for f. So for example, one of them, one of them might represent
all of the values of x and y, where f of x,y is equal to, you know, two, right, so if you looked
at all of the values of x and y where this is true, you'd find yourself on one of these lines, and each line represents
a different possible value for what this constant here actually is. So what I'm going to do here, is I'm actually going to just zoom in on one particular contour line, right. So this here is something
that I'm going to vary, where I'm going to be able to change what the constant we're
setting f equal to is, and look at how the contour
line changes as a result. So for example, if I
put it around here-ish, what you're looking at
is the contour line, the contour line for f of x,y equals 0.1. So all of the values on
these two blue lines here tell you what values
of x and y satisfy 0.1. But on the other hand, I
could also shift this guy up, and maybe I'll shift it up, I'm going to set to where that constant is actually equal to one, so that would be kind of an alternative, we'll just kind of separate over here. That would be the line where f of x,y is set equal to one, itself. And the main thing I
want to highlight here, is that at some values, like 0.1, this contour line
intersects with the circle, it intersects with our constraint. And let's just thing
about what that means, if there's a point, x and y, on that intersection there, that basically gives us a
pair of numbers, x and y, such that, this is true, that fact that f of x,y equals 0.1, and also that x squared
plus y squared equals one. So it means this is something that actually exists and is possible. And in fact, we can see
that there is four different pairs of numbers where that's true, where they intersect here, where they intersect over here, and then the other two, kind of symmetrically on that side. But on the other hand, if
we look as this other world, where we shift up to the
line f of x,y equals one, this never intersects with the constraint. So what that means is x,y, the pairs of numbers
that satisfy this guy, are off the constraint, they're off of that circle, x squared plus y squared equals one. So what that tells us, as we try to maximize this function, subject to this constraint, is that we can never get as high as one. 0.1 would be achievable, and in fact, if we kind of go back to
that, and we look at 0.1, if I upped that value, and you know, changed
it to the line where, instead what you're looking at is 0.2, that's also possible, because it intersects with the circle. And in fact, you could
play around with it, and increase it a little bit more, and if I go to 0.3, instead, and I go over here and I say,
0.3, that's also possible. And what we're basically trying to do is find the maximum value
that we can put here, the maximum value so that if we look at the line that represents f
of x,y equals that value, it still intersects with the circle. And the key here, the key observation, is that that maximum value happens when these guys are tangent. And in the next video, I'll start going into the details of how we can use that observation, this notion of tangency,
to solve the problem, to find the actual value of x and y that maximizes this,
subject to the constraint. But in the interim, I kind
of want you to mull on that, and think a little bit about
how you might use that. What does tangency mean here? How can you take advantage
of certain other notions that we've learned about
in multi-variable calculus, like, hint, hint, the gradient, to actually solve something like this. So with that, I will see you next video.